Automatically receive pictures and upload them to the server, suitable for all applications such as WeChat and Enterprise WeChat
Preface
The WorkTool enterprise micro robot can receive messages from customer groups, but receiving pictures has always been a problem. This article takes enterprise WeChat as an example. The actual principle has broad applicability and can be extended to all APPs such as WeChat.
If you want to receive pictures in Qiwei messages, you’d better make sure before reading below:
- You have used WorkTool Enterprise Robot and have accessed third-party QA yourself (or have callback interface development capabilities)
- Have picture OCR or image recognition capabilities or related interfaces
text
Let’s go directly to the access step. First of all, WorkTool Enterprise Micro Robot is made based on the automation framework based on accessibility services. Its screen reading technology can only obtain text content, but cannot obtain streaming media content such as pictures. At this time, there may be two ideas: the first option is that the picture always needs to be downloaded to the local, we directly monitor the local new file generation to achieve the image acquisition; the second option is to automatically take screenshots, when we find the picture, click to view it and then take a screenshot , save the image file yourself. Option 1 has been introduced above ( click to view ). This article describes the implementation of Option 2. The workflow chart is as follows:
WorkTool's self-developed automation framework is implemented in combination with the screen recording/screenshot tool class MediaProjection natively provided by Android. The core classes for screenshots are as follows:
package org.yameida.worktool.activity
import android.app.Activity
import android.content.Context
import android.content.Intent
import android.content.pm.PackageManager
import android.graphics.Bitmap
import android.graphics.Color
import android.graphics.PixelFormat
import android.hardware.display.DisplayManager
import android.media.Image
import android.media.ImageReader
import android.media.projection.MediaProjectionManager
import android.os.Bundle
import android.os.Handler
import android.os.Looper
import android.os.SystemClock
import androidx.appcompat.app.AppCompatActivity
import com.blankj.utilcode.constant.PermissionConstants
import com.blankj.utilcode.util.*
import org.yameida.worktool.utils.capture.AndroidUtils
import org.yameida.worktool.utils.capture.MediaProjectionHolder
import android.util.DisplayMetrics
import android.view.WindowManager
import org.yameida.worktool.utils.startServiceSafe
import org.yameida.worktool.service.PlayNotifyService
import org.yameida.worktool.service.fastStartActivity
import java.lang.Exception
/**
* Created by Gallon on 2019/7/30.
*/
class GetScreenShotActivity : AppCompatActivity() {
private var mediaProjectionManager: MediaProjectionManager? = null
private var hideFloatWindow = false
private val handler = Handler(Looper.getMainLooper())
companion object {
val HIDE_FLOAT_WINDOW = "hideFloatWindow"
fun startCapture(): Bitmap? {
if (MediaProjectionHolder.mMediaProjection == null) {
LogUtils.e("截图失败 mediaProjection未初始化")
fastStartActivity(Utils.getApp(), GetScreenShotActivity::class.java)
return null
}
val imageReader = ImageReader.newInstance(ScreenUtils.getScreenWidth(), ScreenUtils.getScreenHeight(), PixelFormat.RGBA_8888, 1)
val virtualDisplay = MediaProjectionHolder.mMediaProjection?.createVirtualDisplay("ScreenShout",
ScreenUtils.getScreenWidth(), ScreenUtils.getScreenHeight(), ScreenUtils.getScreenDensityDpi(),
DisplayManager.VIRTUAL_DISPLAY_FLAG_AUTO_MIRROR,
imageReader.surface, null, null)
var image: Image? = null
var tryCount = 0
while (tryCount < 10 && image == null) {
SystemClock.sleep(250)
image = imageReader.acquireNextImage()
}
if (image == null) {
LogUtils.i("GetScreenShotActivity", "image is null.")
return null
}
val width = image.width
val height = image.height
val planes = image.planes
val buffer = planes[0].buffer
val pixelStride = planes[0].pixelStride
val rowStride = planes[0].rowStride
val rowPadding = rowStride - pixelStride * width
var bitmap = Bitmap.createBitmap(width + rowPadding / pixelStride, height, Bitmap.Config.ARGB_8888)
bitmap.copyPixelsFromBuffer(buffer)
bitmap = Bitmap.createBitmap(bitmap, 0, 0, width, height)
image.close()
imageReader.close()
virtualDisplay?.release()
return bitmap
}
}
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
LogUtils.i("onCreate")
window.statusBarColor = Color.TRANSPARENT
mediaProjectionManager = Utils.getApp().getSystemService(Context.MEDIA_PROJECTION_SERVICE) as MediaProjectionManager
hideFloatWindow = intent.getBooleanExtra(HIDE_FLOAT_WINDOW, false)
PermissionUtils.permission(PermissionConstants.STORAGE)
.callback(object : PermissionUtils.SimpleCallback {
override fun onGranted() {
LogUtils.d("start record")
mediaProjectionManager?.apply {
val intent = this.createScreenCaptureIntent()
if (packageManager.resolveActivity(intent, PackageManager.MATCH_DEFAULT_ONLY) != null) {
startActivityForResult(intent, 0)
} else {
ToastUtils.showShort("抱歉，你的手机暂不支持录屏")
}
}
}
override fun onDenied() {
ToastUtils.showShort("请允许申请的权限，否则无法录屏")
finish()
}
})
.request()
}
override fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent?) {
super.onActivityResult(requestCode, resultCode, data)
LogUtils.i("onActivityResult")
if (resultCode == Activity.RESULT_OK && data != null) {
LogUtils.e("mediaProjectionManager: $mediaProjectionManager")
LogUtils.e("resultCode: $resultCode")
LogUtils.e("data: $data")
try {
val mWindowManager = getSystemService(WINDOW_SERVICE) as WindowManager
val metrics = DisplayMetrics()
mWindowManager.defaultDisplay.getMetrics(metrics)
} catch (e: Exception) {
LogUtils.e("MediaProjection error")
}
sendBroadcast(Intent())
val service = Intent(this, PlayNotifyService::class.java)
service.putExtra("setMediaProject", true)
service.putExtra("code", resultCode)
service.putExtra("data", data)
startServiceSafe(service)
}
finish()
}
}
Click the picture to enter the picture browsing page and call the screenshot startCapture method to get the bitmap. Of course, we can cache the bitmap data size. If it is found to be inconsistent with the last time, we can think that a new picture file has been obtained and then push it to us. On the target server, the background will process and analyze the image and then selectively reply (call WorkTool to send a message command).
API document address of WorkTool Qiwei Robot : https://worktool.apifox.cn/
Summarize
At this point, you should understand the principle of the automated framework to achieve image acquisition. However, this solution still has instability. For example, the screenshot may cause the consistency judgment of the two images to become invalid due to other information appearing on the screen. Moreover, the speed of this solution It is indeed slower. We are investigating whether there is a better way or optimizing its speed and stability. WorkTool Qiwei Robot can automatically push the received pictures to the third-party system. If you haven’t learned about WorkTool yet, you can check the official website or the introductory tutorial to try it out.