Huawei Cloud OBS helps IoT data forwarding and storage

Wuhan Yuan Chuanghui Returns, Let’s Talk About Large Models on April 20th”

This article is shared from Huawei Cloud Community " Unlocking a new realm of IoT data management: Huawei Cloud OBS helps data forwarding and storage [Play with Huawei Cloud IoT with zero code and zero hardware] ", author: Y-StarryDreamer.

1. Background introduction

The rapid development of Internet of Things (IoT) technology is driving the arrival of an intelligent society. In this digital era, with the increasing number of IoT devices and the rapid growth of data scale, how to effectively store and manage these massive data has become an important challenge. As a leading cloud computing service provider, Huawei Cloud actively responds to this challenge and provides users with a wealth of IoT solutions, including Object Storage Service (OBS).

As a highly scalable, safe and reliable cloud storage service, OBS provides users with flexible and efficient data storage solutions. It adopts a distributed storage architecture that can easily cope with the storage needs of massive data, and provides a variety of data protection and security control functions to ensure that user data is safe and reliable.

In IoT applications, long-term storage and management of data is crucial. Traditional data storage methods often require users to build and maintain servers by themselves, which is costly and complex to manage. Forwarding data to OBS has become a more convenient and economical solution. By storing data in OBS, users can take full advantage of its high reliability and low cost without worrying about data loss or insufficient storage space. At the same time, OBS also provides rich data management and analysis functions to help users better understand and utilize data to support business decisions.

In this article, we will delve into how to forward the data reported by Huawei cloud devices to OBS for long-term storage. First, we will introduce how to create an OBS bucket and discuss the bucket creation process and related configuration options in detail. Next, we will focus on how to configure device access services, including steps such as creating data forwarding rules and setting forwarding targets. Finally, we will demonstrate through practical operations how to verify the effectiveness of data forwarding and how to use OBS management tools to view and manage stored data.

By reading this article, readers will have a comprehensive understanding of the application scenarios and advantages of OBS in Huawei Cloud IoT solutions, and master the operation method of forwarding device data to OBS for long-term storage, thereby providing technical support and support for the development and deployment of IoT applications. guide.

2. Introduction to Object Storage Service (OBS)

Object Storage Service (OBS) is a highly reliable, safe and reliable cloud storage service provided by Huawei Cloud. OBS adopts a distributed architecture and has the characteristics of high availability, high reliability and high scalability. Users can easily upload, download and manage data stored in OBS through a simple API interface or management console.

Key features of OBS include:

High reliability and availability : OBS uses multi-copy storage and intelligent disaster recovery mechanisms to ensure data security and reliability. Whether it is a hardware failure, network failure or other abnormal situation, OBS can ensure the persistence and availability of data, ensuring that user data can be accessed at any time.
Powerful data management functions : OBS provides rich data management functions, including data classification, life cycle management, data migration, etc. Users can flexibly manage and organize data stored in OBS according to their business needs, thereby improving the value and utilization efficiency of data.
Flexible bucket management : Users can create and manage multiple buckets according to their own needs, and control permissions on the buckets. Each bucket has an independent namespace and access control policy. Users can configure and manage the bucket according to actual conditions to ensure data security and privacy.

OBS plays an important role in the Huawei Cloud ecosystem, providing users with efficient and reliable data storage solutions that are widely used in various scenarios, including data backup, archiving, multimedia storage, etc. Whether individual users, enterprise users or developers, they can use OBS to easily store and manage data, thereby better supporting business development and innovative applications.

3. Create an OBS bucket

Before forwarding the data reported by the device to OBS, you first need to create an OBS bucket. OBS buckets are the basic unit for storing data. Users can create multiple buckets according to their own needs, and configure and manage each bucket.

Steps to create an OBS bucket:

Log in to the official Huawei Cloud website and access the object storage service

Open your browser, go to the official Huawei Cloud website, and log in to your account. Find and click the "Object Storage Service" link in the console to enter the OBS service page.
Enter the object storage service management console

On the OBS service page, you will see an "Enter console" button. Clicking this button will take you to the management console page of the object storage service.
Create a new OBS bucket

In the upper right corner of the management console page, you will find a "Create Bucket" button. Click this button and the system will guide you to the interface for creating an OBS bucket.
Fill in the bucket configuration information

In the bucket creation interface, you need to fill in some configuration information about the new bucket, including the bucket name, region, storage type, etc. Choose the appropriate configuration option based on your needs and application scenarios.
Confirm and create

After filling in the bucket configuration information, you can preview the configuration of the bucket you created. After confirming that everything is correct, click the "Create Now" button and the system will start creating a new OBS bucket.
Wait for creation to complete

The process of creating a bucket may take some time, depending on the configuration you choose and the current system load. Please wait patiently until the system prompts that the bucket creation is complete.

4. Configure device access service

In this article, we will introduce in detail how to use Python code to forward the data reported by the device to Huawei Cloud Object Storage Service (OBS) for long-term storage. We will combine sample code and detailed explanations to help readers fully understand the deployment process and code implementation details.

* Configure Huawei Cloud Object Storage Service (OBS)

First, we need to create an Object Storage Service (OBS) bucket on the Huawei Cloud Platform to store data reported by the device. In the Huawei Cloud console, enter the object storage service management page and follow the following steps to create an OBS bucket:

Click "Enter Console" to enter the object storage service management console.
Click the "Create Bucket" button in the upper right corner of the page.
Select the bucket specifications and configuration items according to your needs, and then click the "Create Now" button to create a new OBS bucket.

*Write Python deployment code

Next, we will write Python code to implement the function of forwarding device data to the OBS bucket. The following is a detailed code explanation and deployment process:

import logging
from obs import ObsClient

# Configure Huawei Cloud OBS parameters
access_key_id = 'your_access_key_id'
secret_access_key = 'your_secret_access_key'
server = 'your_obs_server' # The address of the OBS service, for example: obs.cn-north-4.myhuaweicloud.com

# Configure device access information
device_id = 'your_device_id'
device_secret = 'your_device_secret'

# Configure data forwarding rules
topic = 'your_topic' # Topic of data reported by the device
obs_bucket = 'your_obs_bucket' # OBS bucket name

#Configuration log
logging.basicConfig(level=logging.INFO)

#Create OBS client
obs_client = ObsClient(access_key_id, secret_access_key, server)

#Create device access service and simulate device reporting data
def report_data_to_obs(data):
    try:
        #Report data to OBS
        response = obs_client.putContent(obs_bucket, topic, data)
        if response.status < 300:
            logging.info("Data reported to OBS successfully.")
        else:
            logging.error("Failed to report data to OBS. Status code: %d" % response.status)
    except Exception as e:
        logging.error("An error occurred while reporting data to OBS: %s" % str(e))

# Simulate device reporting data
def simulate_device_data():
    data = {'temperature': 25, 'humidity': 60, 'pressure': 1013} # Simulate the data reported by the device
    report_data_to_obs(data)

if __name__ == "__main__":
    simulate_device_data()

In the above code:

We use a package to interact with Huawei Cloud's Object Storage Service (OBS), make sure you have the package installed. obs
Please replace the , and parameters in the code with the specific information of your Huawei Cloud OBS account. access_key_idsecret_access_key server
report_data_to_obs The function is used to upload the data reported by the device to the specified OBS bucket.
simulate_device_data The function is used to simulate the data reported by the device. You can change the data content according to the actual situation.

* Deploy and run code

After configuring the parameters in the code and saving it, you can deploy the code to your server or local environment and run the Python script. After running, you will see success or failure information printed in the log, and whether the data was successfully uploaded to the OBS bucket.

Through the above steps, you have successfully forwarded the data reported by the device to Huawei Cloud Object Storage Service and achieved long-term storage of the data. This solution can help you manage and analyze large amounts of IoT data and provide reliable data support for your applications. We can clearly understand the structure of the entire code and the functions of each part. This code implements the function of forwarding data reported by the device to Huawei Cloud Object Storage Service. Through reasonable configuration and invocation, reliable storage and management of data can be achieved in IoT applications.

5. Verification operation

After configuring and activating the data forwarding rules, you need to verify whether the data forwarding is successful and ensure that the data reported by the device can be correctly stored in the specified OBS bucket.

Steps to verify data forwarding:

Report data using a real device or simulator

First, you can report data through a real device or a simulated device to verify whether the data forwarding function works properly. Make sure the device can connect to the device access service normally and report data successfully. You can use a device's unique identifier (such as a device ID) to ensure that data is correctly identified and processed.

Query reported data

Log in to the OBS management console of Huawei Cloud and select the corresponding OBS bucket in the console to view the object list or object details in the bucket. You should be able to see objects in the bucket that correspond to the data reported by the device. Confirm that the object list contains the data objects you expect to store, and that the contents of the objects are consistent with the data reported by the device.

Through the above steps, you can verify whether the data forwarding function is normal and ensure that the data reported by the device can be successfully stored in the designated OBS bucket. If you can find and confirm the data reported by the device in the OBS bucket, then the data forwarding function has been successfully configured and is working properly.

The following is sample code to verify data forwarding using the Python SDK:

from obs import ObsClient

#Initialize ObsClient
obs_client = ObsClient(
    access_key_id='your_access_key_id',
    secret_access_key='your_secret_access_key',
    server='your_obs_endpoint' # The endpoint of the OBS service, such as obs.cn-north-1.myhuaweicloud.com
)

#Query the object list in the specified bucket
def list_objects(bucket_name):
    try:
        resp = obs_client.listObjects(bucket_name)
        if resp.status == 200:
            return resp.body.contents
        else:
            print(f'Error: {resp.status}')
            return None
    except Exception as e:
        print(f'Error: {e}')
        return None

# Query the details of the specified object
def get_object_detail(bucket_name, object_key):
    try:
        resp = obs_client.getObjectMetadata(bucket_name, object_key)
        if resp.status == 200:
            return resp.body
        else:
            print(f'Error: {resp.status}')
            return None
    except Exception as e:
        print(f'Error: {e}')
        return None

if __name__ == "__main__":
    # Specify the OBS bucket name and the object key (key) reported by the device
    bucket_name = 'your_obs_bucket_name'
    object_key = 'your_object_key'

    #Query object list
    objects = list_objects(bucket_name)
    if objects:
        print(f'Objects in bucket {bucket_name}: {objects}')

        # Query object details
        object_detail = get_object_detail(bucket_name, object_key)
        if object_detail:
            print(f'Object detail: {object_detail}')
        else:
            print(f'Failed to get object detail for key {object_key}')
    else:
        print(f'Failed to list objects in bucket {bucket_name}')

In the above code you need to replace the following:

your_access_key_id: Your Huawei Cloud Access Key ID.
your_secret_access_key: Your Huawei Cloud Secret Access Key.
your_obs_endpoint: The endpoint of your OBS service, such as obs.cn-north-1.myhuaweicloud.com.
your_obs_bucket_name: Your OBS bucket name.
your_object_key: The object key reported by the device, used to query object details.

Running the above code will query the object list in the specified bucket and obtain the details of the specified object to verify whether the data forwarding is successful.

6. Best practices and considerations

When forwarding data to OBS, there are some best practices and precautions that need to be noted:

Best Practices

Data encryption and security :

For sensitive data, it is recommended to encrypt data during data transmission to ensure data security. You can use secure encryption protocols such as HTTPS to protect data during transmission.

Data lifecycle management :

According to the life cycle of data and business needs, reasonably set the storage period and expiration policy of data to avoid storing expired or useless data. Regularly review and cleanse data that is no longer needed to free up storage space and reduce storage costs.

Monitoring and alarming :

It is recommended to monitor and alarm the data forwarding and storage process, detect and handle abnormal situations in a timely manner, and ensure the reliability and stability of the data. You can set monitoring indicators and alarm rules, monitor the status and performance of data forwarding, and take timely measures to solve problems.

Precautions

Bucket permission settings :

When creating a bucket, you need to set the bucket's permissions and access policies according to actual needs to ensure data security and compliance. You can restrict access to a bucket to allow only specific users or entities to access the bucket, and adopt access control policies to protect the data in the bucket.

Region selection :

When setting the forwarding target, you need to select the same region as the OBS service to ensure that data can be forwarded and stored in the specified OBS bucket smoothly. Selecting the same area can reduce the delay of data transmission and improve the efficiency and reliability of data transmission.

7.THE end

This article details how to forward data reported by Huawei cloud devices to Object Storage Service (OBS), and highlights the importance and advantages of this operation. By storing data in OBS, users gain the following advantages:

High reliability and security : OBS adopts distributed architecture and multi-copy storage to ensure the safety and reliability of data. It also provides security functions such as data encryption to ensure the confidentiality and integrity of data.
Low-cost data storage : OBS provides an affordable storage solution. Users only need to pay according to the actual amount of data stored, avoiding the need to purchase storage equipment and maintenance costs in advance in traditional storage solutions.
Powerful data management functions : OBS provides rich data management functions, including data classification, life cycle management, data migration, etc., to help users better manage and utilize the data stored in OBS.

By forwarding the data reported by the device to OBS, users can achieve long-term storage and management of data, providing stable and reliable data support for applications. Therefore, we encourage readers to try and explore more data storage and management methods to improve the efficiency and reliability of applications. By rationally utilizing cloud storage services, users can better cope with the growing volume of data and complex data management needs, providing stronger support for the development of IoT applications.

Click to follow and learn about Huawei Cloud’s new technologies as soon as possible~