Use pandas to operate Oracle and MySQL databases

A large part of Python's powerful data processing ability comes from Pandas. Pandas is not limited to reading local offline files, but also can read database data online, and then write it back to the database after processing. Pandas mainly establishes a link with the database in sqlalchemy mode, and supports mainstream databases such as Mysql, postgresql, Oracle, MS SQLServer, and SQLite.

SQLAlchemy

SQLAlchemy is an open source software under the Python programming language. Provides SQL toolkit and object-relational mapping (ORM) tools, issued under the MIT license. You can install the SQLAlchemy module using the pip command:

pip install sqlalchemy

The SQLAlchemy module provides the create_engine() function to initialize the database connection. SQLAlchemy uses a string to represent the connection information:

'Database type+database driver name://username:password@machine address:port number/database name'

Pandas read and write MySQL database

We need the following three libraries to implement Pandas to read and write MySQL databases:

• pandas
• pymysql
• sqlalchemy

Create a database connection:

connect = create_engine('mysql+pymysql://username:password@host:port/dbname')

Read the table in the MySQL database as a DataFrame in pandas:

sql = 'select * from test'
data = pd.read_sql(sql,connect)

Store the DataFrame in pandas to the MySQL database:

data.to_sql('tablename', connect, index=False, if_exists='append')

Among them, if_exists has three parameters,'fail': if the table exists, the deposit fails

                                               'replace': If the table exists, replace

                                               'If the table exists', write additional

Pandas read and write Oracle database

We need the following three libraries to implement Pandas to read and write MySQL databases:

• pandas
• cx_Oracle
• sqlalchemy

Create database connection:

connect=create_engine('oracle://username:password@host:port/sid')

Read the table in the Oracle database as a DataFrame in pandas:

sql = 'select * from test'
data = pd.read_sql(sql,connect)

Store the DataFrame in pandas to the Oracle database:

data.to_sql('tablename', connect, index=False, if_exists='append')