Oracle table level real-time synchronization & bi-directional synchronization test program

1. Test target
2. Test Environment
2.1 test database
2.2 Create Test Table 
2.3 Open Database Archiving 
2.4 s3 user to establish synchronization 
2.5 Configuration Synchronization task
2.5.1 installed beedi 
2.5.2 total amount Task 
Task 2.5.3 increments  
3. Test Procedure  
3.1 total amount synchronization  
3.2 real-time synchronization 
3.2.1 additions and deletions to merge 
3.2.2 additions and deletions to distribution
3.2.3 bulk insert
3.2.4 insert a large transaction
3.2.5 update sequence 
3.2.6 disconnection
3.2.7 concurrent update
3.2.8 batch update
3.2.9 large transaction update 

 

1. Test objective
and more real-time data synchronization to the Group subsidiaries central database, and then modify the data specified in the Group Center, the library, the modified data in real time to push back the corresponding branch of the database.


2. Test Environment


2.1 Test database
three Oracle Server s1, s2 and s3, where s1 and s2 representative of sub-library, s3 Representative repository. 


2.2 Create Test Table
Create Test Table t_share_pool_s1 server S1
Create scott.t_share_pool_s1 Table (ID NUMBER (15) the NOT NULL, COMPANY_CODE VARCHAR2 (10) the NOT NULL, WORK_F1 VARCHAR2 (10), WORK_F2 VARCHAR2 (10), Last_time DATE, a PRIMARY KEY (ID, COMPANY_CODE))
is inserted into the test data
      DECLARE 
      count2 int: =. 1;
      the begin
      the while count2 <= 1000 Loop
            iNSERT INTO scott.t_share_pool_s1 (ID, COMPANY_CODE, WORK_F1, WORK_F2, Last_time) values (count2, 'Sl', 'w1_0' , 'W2 - 0', SYSDATE);
           count2: = count2 +. 1;
       End Loop;
the commit;
       End;
Create test table server s2 t_share_pool_s2
create table scott.t_share_pool_s2(ID NUMBER(15) NOT NULL, COMPANY_CODE VARCHAR2(10) NOT NULL,WORK_F1 VARCHAR2(10),WORK_F2 VARCHAR2(10),LAST_TIME DATE,PRIMARY KEY(ID,COMPANY_CODE))
插入测试数据
      declare 
      count2 int := 1;
      begin
      while count2<=1000 loop
insert into scott.t_share_pool_s2 (ID, COMPANY_CODE, WORK_F1, WORK_F2, LAST_TIME) values(count2,'S2','w1_0','w2_0',sysdate);
           count2 := count2+1;
       end loop;
commit;
       end;

S3 test table created in the server t_share_pool
Create Table scott.t_share_pool (ID NUMBER (15) the NOT NULL, COMPANY_CODE VARCHAR2 (10) the NOT NULL, WORK_F1 VARCHAR2 (10), WORK_F2 VARCHAR2 (10), Last_time DATE, a PRIMARY KEY (ID, COMPANY_CODE ))


2.3 Open Database Archiving
sequentially s1, s2, s3 database following the SQL
sqlplus / AS SYSDBA
the shutdown immediate;
Startup Mount;
ALTER Database ARCHIVELOG;
ALTER Open Database;
ALTER Database log the Add Supplemental Data (Primary Key, UNIQUE index) Columns;


2.4    s3建立同步用户
-- Create the user 
create user BEEDI identified by BEEDI default tablespace USERS  temporary tablespace TEMP;
-- Grant/Revoke role privileges 
grant connect to BEEDI;
grant resource to BEEDI;
-- Grant/Revoke system privileges 
grant delete any table to BEEDI;
grant insert any table to BEEDI;
grant select any table to BEEDI;
grant unlimited tablespace to BEEDI;
grant update any table to BEEDI;


2.5 Configuration Synchronization task


2.5.1 Installation BeeDI
run BeeDI SE installer, select [Next] button until the pop-up software registration window, shown as follows:
 
Click the window button to bring up the software registration dialog box, shown as follows:
 
Click the button Find. import BeeDI authorization file, then select [OK] button to close the [software registration] dialog box, the Setup Wizard pop-up screen shown below:
 
select the software after installation directory, click [Next] button, the installation program begins copying files to the installation directory, after the completion of the installation window appears, as shown:
 
after starting beedi, login dialog box, as shown in FIG.
 
Tip: BeeDI installed, create two default user admin and guest, and their passwords are empty.
After entering the correct user and password and click [OK], enter BeeDI


2.5.2 task total amount
total amount of synchronization task for s1 and s2 merged into the existing data synchronization database s3, as follows:
select the Toolbars [new file] button [New Job] dialog box [Job Name] at input sync_full , [type] select ETL template, click [OK] button to create a job file.
Successively establishing s1, s2, s3,, wherein the database connection s1 and s2 may use the sys connection, s3 used beedi connection. Released from the node table scott.t_share_pool s1 to s3 scott.t_share_pool_s1 node table drag the mouse, and then releases the mouse drag s2 from the table to the node table node scott.t_share_pool scott.t_share_pool_s2 s3, the results are as follows:
 
select the Toolbars [ start] button to execute the job sync_full, results are as follows:
 
 


2.5.3    增量任务
增量同步任务用于实时同步分库数据变化到中心库,同时对于中心库的数据变化,根据数据来源推回到对应分库。
选择工具栏[新建文件]按钮,在[新建作业]对话框的[作业名称]输入sync_log_up,[类型]选择ETL模板,点击[确定]按钮创建分库到中心库的增量同步作业文件。
依次建立到s1、s2、s3的数据库连接,其中s1和s2可使用sys连接,s3使用beedi连接。从s1拖拽表节点scott.t_share_pool_s1到s3的表节点scott.t_share_pool上释放鼠标,再从s2拖拽表节点scott.t_share_pool_s2到s3的表节点scott.t_share_pool上释放鼠标,结果如下:
 
点击scott.t_share_pool_s1对应抽取组件窗口的[选项]按钮,弹出[选项]对话框,点击[增量抽取]复选框,弹出[增量抽取设置]对话框,操作如下图:
 
 
 
选择[日志]按钮,拖拽主键列ID和COMPANY_CODE到逻辑主键窗口,[解析点]使用默认时间,取消[分解UPDATE]复选,完成后的界面配置如下:
 
点击[返回]按钮关闭[增量抽取设置]对话框,显示界面如下:
 
点击[确定]按钮关闭抽取组件[选项]对话框。
点击scott.t_share_pool_s2抽取组件窗口的[选项]按钮,重复以上步骤,完成基于日志的增量设置。
选择工具栏[启动]按钮执行作业sync_log_up,结果如下:
 
选择工具栏[新建文件]按钮,在[新建作业]对话框的[作业名称]输入sync_log_down,[类型]选择ETL模板,点击[确定]按钮创建中心库到分库的增量同步作业文件。
依次建立到s3、s1、s2的数据库连接,其中s3、s1和s2可使用sys连接。从s3拖拽表节点scott.t_share_pool到模板建立抽取组件,操作如下:
 
右键点击s1表结点scott.t_share_pool_s1,从快捷菜单选择[装载]命令在模板建立装载组件,操作如下:
 
 
右键点击s2表结点scott.t_share_pool_s2,从快捷菜单选择[装载]命令在模板建立装载组件,操作结果如下:
 
拖拽scott.t_share_pool对应抽取组件的映射拖拽点到scott.t_share_pool_s1对应的装载组件窗口,弹出[全表映射模式设置]对话框,选择[列名匹配映射],点击[确定]按钮完成抽取组件到装载组件的列映射,操作结果如下:
 
 
 
再次拖拽scott.t_share_pool对应抽取组件的映射拖拽点到scott.t_share_pool_s2对应的装载组件窗口,选择[列名匹配映射],建立scott.t_share_pool抽取组件到scott.t_share_pool_s2装载组件的列映射,完成后界面如下:
 
点击scott.t_share_pool对应抽取组件窗口的[选项]按钮,弹出[选项]对话框,点击[增量抽取]复选框,弹出[增量抽取设置]对话框,操作如下图:
 
 
 
选择[日志]按钮,拖拽主键列ID和COMPANY_CODE到逻辑主键窗口,[解析点]使用默认时间,取消[分解UPDATE]复选,[忽略用户]输入beedi,完成后的界面配置如下:
 
点击[返回]按钮关闭[增量抽取设置]对话框,显示界面如下:
 
点击[确定]按钮关闭抽取组件[选项]对话框。
点击scott.t_share_pool_s1对应装载组件窗口的[选项]按钮,弹出[选项]对话框,点击[过滤表达式]输入按钮,弹出[过滤表达式脚本设置]对话框,在其中输入过滤脚本equals(query_15.COMPANY_CODE,"S1"),操作如下图:
(注:此处query_15表示抽取组件名,可用实际组件名替换)
 
 
 
点击[确定]按钮关闭[过滤表达式脚本设置]对话框,显示界面如下:
 
点击[确定]按钮关闭装载组件[选项]对话框。
点击scott.t_share_pool_s2装载组件窗口的[选项]按钮,重复以上操作设置过滤表达式equals(query_15.COMPANY_CODE,"S2"),结果如下。
(注:此处query_15表示抽取组件名,可用实际组件名替换)
 
选择工具栏[启动]按钮执行作业sync_log_down,结果如下:
 
右键点击调度窗口的作业sync_full,在快捷菜单中选择[删除]命令将该作业从调度窗口删除。
 
 
使用CTRL键,选中调度窗口的作业sync_log_up和sync_log_down,  


选择 [定时]按钮,弹出[作业定时设置]对话框,点击[确定]按钮,操作结果如下图:
 
 
 


3.    测试步骤


3.1    全量同步
依次将s1和s2表数据同步到s3,查询观察中心库数据是否一致。
作业sync_full执行成功后,s1和s2分库数据与中心库数据一致


3.2    实时同步


3.2.1    增删改合并
分别对s1和s2的表进行5次增删改操作,查询观察中心库数据是否一致。
declare 
count2 int := 1001;
begin
while count2<=1005 loop
insert into scott.t_share_pool_s1 (ID, COMPANY_CODE, WORK_F1, WORK_F2, LAST_TIME) values(count2,'S1','w1_0','w2_0',sysdate);
update scott.t_share_pool_s1 set work_f1 = 'w1_88',work_f2='w2_88' where ID = count2 and COMPANY_CODE = 'S1';
delete from scott.t_share_pool_s1  where ID = count2 and COMPANY_CODE = 'S1';
count2 := count2+1;
end loop;
commit;
end;

declare 
count2 int := 1001;
begin
while count2<=1005 loop
insert into scott.t_share_pool_s2 (ID, COMPANY_CODE, WORK_F1, WORK_F2, LAST_TIME) values(count2,'S2','w1_0','w2_0',sysdate);
update scott.t_share_pool_s2 set work_f1 = 'w1_88',work_f2='w2_88' where ID = count2 and COMPANY_CODE = 'S2';
delete from scott.t_share_pool_s2  where ID = count2 and COMPANY_CODE = 'S2';
count2 := count2+1;
end loop;
commit;
end;


3.2.2    增删改分发
分别修改中心库对应s1和s2的5条数据,查询观察s1和s2数据是否一致。
declare 
count2 int := 1;
begin
while count2<=5 loop
update scott.t_share_pool set work_f1 = 'w1_99',work_f2='w2_99' where ID = count2 and COMPANY_CODE = 'S1';
count2 := count2+1;
end loop;
commit;
end;

declare 
count2 int := 1;
begin
while count2<=5 loop
update scott.t_share_pool set work_f1 = 'w1_77',work_f2='w2_77' where ID = count2 and COMPANY_CODE = 'S2';
count2 := count2+1;
end loop;
commit;
end;


3.2.3    批量插入
分别在s1和s2批量插入10000条数据,每500条提交一次,查询观察中心库数据是否一致。
declare 
count2 int := 1001;
begin
while count2<=11000 loop
insert into scott.t_share_pool_s1 (ID, COMPANY_CODE, WORK_F1, WORK_F2, LAST_TIME) 
               values(count2,'S1','w1_0','w2_0',sysdate);
count2 := count2+1;
if mod(count2-1001,500)=0 then
commit;
end if;
end loop;
commit;
end;

declare 
count2 int := 1001;
begin
while count2<=11000 loop
insert into scott.t_share_pool_s2 (ID, COMPANY_CODE, WORK_F1, WORK_F2, LAST_TIME) 
               values(count2,'S2','w1_0','w2_0',sysdate);
count2 := count2+1;
if mod(count2-1001,500)=0 then
commit;
end if;
end loop;
commit;
end;


3.2.4    大事务插入
在s1批量插入100000条数据,10000条提交一次,查询观察中心库数据是否一致。
declare 
count2 int := 11001;
begin
while count2<=111000 loop
insert into scott.t_share_pool_s1 (ID, COMPANY_CODE, WORK_F1, WORK_F2, LAST_TIME) 
               values(count2,'S1','w1_0','w2_0',sysdate);
count2 := count2+1;
if mod(count2-11001,10000)=0 then
commit;
end if;
end loop;
commit;
end;


3.2.5    顺序更新
在s1修改某一记录,然后在中心库修改对应记录,查询观察s1和中心库数据是否一致。
update scott.t_share_pool_s1 set work_f1 = 'w1_99',work_f2='w2_99' where ID = 1001;
commit;

update scott.t_share_pool set work_f1 = 'w1_11',work_f2='w2_11' where ID = 1001 and COMPANY_CODE = 'S1';
commit;

3.2.6    断网
断网修改s1某一记录,恢复网络后查询比较中心库是否一致,断网修改中心库某一记录,恢复网络后查询比较s1是否一致。
update scott.t_share_pool_s1 set work_f1 = 'w1_99',work_f2='w2_99' where ID = 1002;
commit;

update scott.t_share_pool set work_f1 = 'w1_11',work_f2='w2_11' where ID = 1002 and COMPANY_CODE = 'S1';
commit;


3.2.7    并发更新
同时重复5次修改s1和中心库的同一记录,查询观察s1和中心库数据是否一致。
declare 
count2 int := 1;
begin
while count2<=5 loop
update scott.t_share_pool_s1 set work_f1 = 'w1_99_'||count2,work_f2='w2_99_'||count2 where ID = 1003;
commit;
count2 := count2+1;
end loop;
end;

declare 
count2 int := 1;
begin
while count2<=5 loop
update scott.t_share_pool set work_f1 = 'w1_11_'||count2,work_f2='w2_11_'||count2 where ID = 1003 and company_code = 'S1';
commit;
count2 := count2+1;
end loop;
end;


3.2.8    批量更新
修改中心库500条记录,每条提交一次,查询观察s1和中心库数据是否一致。
declare 
count2 int := 1;
begin
while count2<=500 loop
update scott.t_share_pool set work_f1 = 'w1_99_'||count2,work_f2='w2_99_'||count2 where ID = count2 and company_code = 'S1';
commit;
count2 := count2+1;
end loop;
end;


3.2.9    大事务更新
修改中心库20000条记录后一次提交,查询观察s1和中心库数据是否一致。
declare 
count2 int := 1;
begin
while count2<=20000 loop
update scott.t_share_pool set work_f1 = 'w1_7_'||count2,work_f2='w2_7_'||count2 where ID = count2 and company_code = 'S1';
count2 := count2+1;
end loop;
commit;
end;

 

 

发布了11 篇原创文章 · 获赞 0 · 访问量 1954

Guess you like

Origin blog.csdn.net/wahahaman/article/details/104087900