应用场景:统计每日会员注册走势【MongoDB-->MySQL】
从MongoDB读取注册用户的信息,处理,写入MySQL对应表中,用于统计注册的情况!!!
MySQL表 t_member_aggregation
CREATE TABLE `t_member_aggregation` ( `year` INT(11) NOT NULL COMMENT '年份', `month` INT(11) NOT NULL COMMENT '月份', `day` INT(11) NOT NULL COMMENT '日期', `amount` BIGINT(20) NULL COMMENT '关注人数', `aggregation_date` DATETIME NOT NULL, PRIMARY KEY (`year`,`month`,`day`)) ENGINE = InnoDB DEFAULT CHARSET = utf8;
字段: year month day amount aggregation_date
year month day 做主键, 同一天注册的 amount+1, aggregation_date纪录上一次统计的时间,每次统计的时间为 aggregation_date 到 系统当前时间。
INSERT INTO `t_member_aggregation` (`year`, `month`, `day`, `amount`, `aggregation_date`) VALUES (?,?,?,?,now()) ON DUPLICATE KEY UPDATE `amount` = `amount` + 1, `aggregation_date` = now();
<!-- Member Data : from Mongo to MySQL 关注用户统计 --> <bean id="memberQuery" class="com.newjetsoft.genen.batch.util.QueryFactory"> <property name="tableName" value="t_member_aggregation"></property> <property name="jdbcTemplateAggregation" ref="jdbcTemplateAggregation"></property> </bean> <bean id="memberInfoResultItemReader" class="org.springframework.batch.item.data.MongoItemReader" scope="step"> <property name="template" ref="mongoUicTemplate"/> <property name="collection" value="Member"/> <property name="targetType" value="com.newjetsoft.genen.batch.aggregation.member.MemberInfoResult" /> <property name="query" ref="memberQuery"/> <property name="sort"> <util:map id="sort"> <entry key="id" value="ASC"/> </util:map> </property> </bean> <bean id="memberInfoResultItemProcessor" class="com.newjetsoft.genen.batch.aggregation.member.MemberInfoResultItemProcessor" /> <bean id="memberInfoResultItemWriter" class="org.springframework.batch.item.database.JdbcBatchItemWriter" scope="step"> <property name="dataSource" ref="dataSourceAggregation"/> <property name="sql"> <value> <![CDATA[ INSERT INTO `t_member_aggregation` (`year`, `month`, `day`, `amount`, `aggregation_date`) VALUES (?,?,?,?,now()) ON DUPLICATE KEY UPDATE `amount` = `amount` + 1, `aggregation_date` = now(); ]]> </value> </property> <property name="ItemPreparedStatementSetter"> <bean class="com.newjetsoft.genen.batch.aggregation.member.UpdateMemberInfoPreparedStatementSetter"/> </property> </bean>
链接: https://pan.baidu.com/s/1gfLxV0n