The traditional e-commerce product display adopts the form of combination of pictures and texts, the copy introduces the relevant parameters of the product, and the exquisite pictures are used to attract the attention of customers. However, due to the color difference and size inconsistency of graphic product display, consumers will have doubts about the inconsistency between the picture and the real object, and consumers need to consume a lot of energy to read and compare before making a purchase decision.
With the continuous improvement of Internet e-commerce platforms, short videos have gradually become the mainstream form of product display. By shooting short videos, products can be displayed in multiple directions, allowing consumers to understand the parameters and advantages of products in a short period of time, which greatly improves the shopping speed of consumers. However, the disadvantage of short video display is that the display time is short and it is difficult to interact with consumers.
The development of 3D technology has given the e-commerce industry the latest way to display products. 3D models can display products in 360° all-round details, allowing consumers to understand products more intuitively and enhance their confidence in products. Consumers can also interact with products through the screen, and can view products in 360 degrees by dragging with one finger, and freely zoom in to view high-definition details, thereby increasing the probability of ordering.
However, the current effective 3D modeling technology is prohibitive for the majority of developers due to its high cost.
1. High technical threshold: professionals plus professional equipment such as depth cameras.
2. High time cost: Professionals manually complete the production, rendering and adjustment of the model. Completing a low-precision model of a simple object starts in hours, while a high-precision model takes longer.
3. High cost: The professional modeling cost of a single product is high, with an average price of thousands of yuan, and complex models are even more expensive.
HMS Core's 3D modeling service for easy modeling. Users only need to use ordinary RGB cameras to automatically generate 3D geometric models and textures of objects by taking images of objects from different angles. For example, in the scene of physical display in e-commerce, you can use this ability to automatically generate the products you want to display. The model is used for 3D display. Users can zoom in or out of products in 360°, view product details, and provide users with a differentiated purchasing experience.
The 3D object modeling capability is completed by the end-cloud collaboration. The end-side is responsible for collecting RGB images, and shooting multiple images around the object to obtain images of different angles of the object. After shooting, upload it to the cloud to realize 3D object modeling. The process and key technologies of cloud modeling include target detection and segmentation, feature detection and matching, sparse point cloud computing, dense point cloud computing, and texture reconstruction.
Show results
Taking bread as an example, simply take multiple images around the bread to get a realistic 3D model of the bread. How is this function achieved? The following are the detailed development steps.
Preparation before development
- Configure the integrated SDK package
In the build.gradle file of the application, add the SDK dependencies of the 3D modeling service in dependencies
// 3D Modeling Kit SDK
implementation 'com.huawei.hms:modeling3d-object-reconstruct:1.0.0.300'
- Configure AndroidManifest.xml
Open the AndroidManifest.xml file in the main folder, you can configure read and write mobile phone storage and camera permissions according to the scene and usage needs, add it before <application>
<!-- 往sdcard中写入数据的权限 -->
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
<!-- 使用相机的权限 -->
<uses-permission android:name="android.permission.CAMERA" />
Development steps
- To use the cloud-side service capability, you need to use the api_key value in "agconnect-services.json", and set the application authentication information through api_key or AccessToken during application initialization. AccessToken has a higher priority. Both methods can be initialized and set once when the application starts, and there is no need to set it multiple times.
(1) Set the AccessToken through the setAccessToken method.
ReconstructApplication.getInstance().setAccessToken("your AccessToken");
(2) Set api_key through the setApiKey method. When you register your app on AppGallery Connect, your app is assigned an api_key.
ReconstructApplication.getInstance().setApiKey("your api_key");
- Create a new 3D object modeling engine and initialize it, and create a new 3D object modeling configurator.
// 新建3D物体建模引擎。
Modeling3dReconstructEngine modeling3dReconstructEngine = Modeling3dReconstructEngine.getInstance(context);
// 新建3D物体建模配置器。
Modeling3dReconstructSetting setting = new Modeling3dReconstructSetting.Factory()
// 设置工作模式为图片模式。
.setReconstructMode(Modeling3dReconstructConstants.ReconstructMode.PICTURE)
// 设置贴图模式为普通模式或PBR模式。
.setTextureMode(Modeling3dReconstructConstants.TextureMode.PBR)
.create();
```
3. 新建上传监听器回调，用于处理拍摄的物体图片上传结果。
private Modeling3dReconstructUploadListener uploadListener = new Modeling3dReconstructUploadListener() { @Override public void onUploadProgress(String taskId, double progress, Object ext) { // Upload progress. } @Override public void onResult(String taskId, Modeling3dReconstructUploadResult result, Object ext) { // Upload successfully processed. } @Override public void onError(String taskId, int errorCode, String message) { // Upload failure handling. } };
4. 使用3D物体建模配置器初始化任务，并且给新建的3D物体建模引擎设置上传监听器，上传采集的图片数据。
// Use the 3D object modeling configurator to initialize the task (this interface needs to be called in a child thread). Modeling3dReconstructInitResult modeling3dReconstructInitResult = modeling3dReconstructEngine.initTask(setting); String taskId = modeling3dReconstructInitResult.getTaskId(); // Set the upload listener. modeling3dReconstructEngine.setReconstructUploadListener(uploadListener); // Call the upload interface of the 3D modeling engine to upload the collected image data. modeling3dReconstructEngine.uploadFile(taskId, filePath);
5. 查询3D物体建模任务状态。
// Querying the 3D object modeling task status requires initializing the task processing class. Modeling3dReconstructTaskUtils modeling3dReconstructTaskUtils = Modeling3dReconstructTaskUtils.getInstance(context); // Call the query interface to get the 3D object modeling task status (this interface needs to be called in a child thread). Modeling3dReconstructQueryResult queryResult = modeling3dReconstructTaskUtils.queryTask(taskId); // Get the modeling task status. int status = queryResult.getStatus();
6. 新建侦听器回调，调用预览函数，预览3D模型
Modeling3dReconstructPreviewListener previewListener = new Modeling3dReconstructPreviewListener() { @Override public void onResult(String taskId, Object ext) { // 3D object modeling preview result. } @Override public void onError(String taskId, int errorCode, String message) { // Preview the error callback function. } }; // Preview model configuration. Modeling3dReconstructPreviewConfig config = new Modeling3dReconstructPreviewConfig.Factory().setTextureMode(Modeling3dReconstructConstants.TextureMode.PBR).create(); // Preview the model. modeling3dReconstructEngine.previewModelWithConfig(taskId, context,config, previewListener);
7. 新建下载监听器回调，用于处理3D物体建模模型文件的下载结果。
private Modeling3dReconstructDownloadListener modeling3dReconstructDownloadListener = new Modeling3dReconstructDownloadListener() { @Override public void onDownloadProgress(String taskId, double progress, Object ext) { // Download progress. }
@Override public void onResult(String taskId, Modeling3dReconstructDownloadResult result, Object ext) { // Download successfully processed. } @Override public void onError(String taskId, int errorCode, String message) { // Download failure handling. } };
8. 新建的下载配置项并将新建的下载监听器传入新建的3D物体建模引擎，下载重建成功的模型文件。
// Set download configuration items. Modeling3dReconstructDownloadConfig downloadConfig = new Modeling3dReconstructDownloadConfig.Factory() // Configure OBJ or glTF format. .setModelFormat(Modeling3dReconstructConstants.ModelFormat.OBJ) // Configure normal or PBR mode. .setTextureMode(Modeling3dReconstructConstants.TextureMode.PBR) .create(); // Set the download listener. modeling3dReconstructEngine.setReconstructDownloadListener(modeling3dReconstructDownloadListener); // Call the download interface of the 3D object modeling engine, pass in the task id, download address and download configuration items, and download the model file. modeling3dReconstructEngine.downloadModelWithConfig(taskId, savePath, downloadConfig);
完成以上步骤就可以在电商应用中实现3D建模能力，用手机就能快速给商品建立3D模型，感兴趣的开发者们可以马上动手体验！
除了商品3D建模和展示，HMS Core电商解决方案还提供了AR交互式购物能力，可实现商品试穿功能，为消费者创造虚拟与现实融合的购物体验，相关内容会在下一期进行详细讲解，敬请期待。
**了解更多详情>>**
访问[华为开发者联盟官网](http://developer.huawei.com/consumer/cn/hms?ha_source=hms1)
获取[开发指导文档](http://developer.huawei.com/consumer/cn/doc/development?ha_source=hms1)
华为移动服务开源仓库地址：[GitHub](http://github.com/HMS-Core)、[Gitee](http://gitee.com/hms-core)
**关注我们，第一时间了解 HMS Core 最新技术资讯~**