當前位置：首頁 > 编程语言 > asp.net >内容正文

asp.net

java mongo 获取所有数据库_Spring Batch —从XML读取并写入Mongo

發布時間：2023/12/10 asp.net 28 豆豆

生活随笔收集整理的這篇文章主要介紹了 java mongo 获取所有数据库_Spring Batch —从XML读取并写入Mongo 小編覺得挺不錯的,現在分享給大家,幫大家做個參考.

Java面試必備之JVM+GC教程

這幾天閑著在優銳課的java學習必備中學習了，在本文中，了解如何使用Spring Batch通過StaxEventItemReader使用ItemReader讀取XML文件并將其數據寫入NoSQL。

在本文中，我們將向展示如何使用Spring Batch使用StaxEventItemReader和ItemReader讀取XML文件，以及如何使用帶有JpaRepository的Custom ItemWriter將其數據寫入NoSQL。在這里，我們使用了MongoDB。

自定義ItemReader或ItemWriter是一個類，我們在其中編寫自己的讀取或寫入數據的方式。在Custom Reader中，我們也需要處理分塊邏輯。如果我們的讀取邏輯很復雜并且無法使用spring提供的Default ItemReader進行處理，那么這將很方便。

使用的工具和庫：

1. Maven 3.5+

2. Spring Batch Starter

3. Spring OXM

4. Data Mongodb starter

5. xstream

Maven依賴關系- 需要配置項目。

<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd"><modelVersion>4.0.0</modelVersion><p><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-parent</artifactId><version>2.2.2.RELEASE</version><relativePath ></relativePath> </parent><groupId>com.example</groupId><artifactId>spring-batch-mongodb</artifactId><version>0.0.1-SNAPSHOT</version><name>spring-batch-mongodb</name><description>Demo project for Spring Boot</description><p><java.version>1.8</java.version><maven-jar-plugin.version>3.1.1</maven-jar-plugin.version></properties><dependencies><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-batch</artifactId></dependency><dependency><groupId>org.springframework</groupId><artifactId>spring-oxm</artifactId></dependency><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-data-mongodb</artifactId></dependency><dependency><groupId>com.thoughtworks.xstream</groupId><artifactId>xstream</artifactId><version>1.4.7</version></dependency><dependency><groupId>org.projectlombok</groupId><artifactId>lombok</artifactId><optional>true</optional></dependency><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-test</artifactId><scope>test</scope><exclusions><exclusion><groupId>org.junit.vintage</groupId><artifactId>junit-vintage-engine</artifactId></exclusion></exclusions></dependency><dependency><groupId>org.springframework.batch</groupId><artifactId>spring-batch-test</artifactId><scope>test</scope></dependency><dependency><groupId>com.h2database</groupId><artifactId>h2</artifactId><scope>runtime</scope></dependency></dependencies><build><p><p><groupId>org.springframework.boot</groupId><artifactId>spring-boot-maven-plugin</artifactId></plugin></plugins></build> </project>

CustomerWriter — 這是我們創建的自定義寫入器，用于將客戶數據寫入MongoDB。自定義編寫器也提供執行復雜操作的功能。

package com.example.writer; import java.util.List; import org.springframework.batch.item.ItemWriter; import org.springframework.beans.factory.annotation.Autowired; import com.example.domain.Customer; import com.example.repository.CustomerRepository; public class CustomerWriter implements ItemWriter<Customer>{@Autowiredprivate CustomerRepository customerRepository;@Overridepublic void write(List<? extends Customer> customers) throws Exception {customerRepository.saveAll(customers);} }

CustomerRepository — 這是一個Mongo存儲庫，可與Mongo數據庫進行對話并執行操作以取回數據。

package com.example.repository; import org.springframework.data.mongodb.repository.MongoRepository; import com.example.domain.Customer; public interface CustomerRepository extends MongoRepository<Customer, String>{ }

客戶 —這是一個包含業務數據的Mongo文檔類。

package com.example.domain; import java.time.LocalDate; import javax.xml.bind.annotation.XmlRootElement; import org.springframework.data.annotation.Id; import org.springframework.data.mongodb.core.mapping.Document; import org.springframework.data.mongodb.core.mapping.Field; import lombok.AllArgsConstructor; import lombok.Builder; import lombok.Data; import lombok.NoArgsConstructor; @AllArgsConstructor @NoArgsConstructor @Builder @Data @XmlRootElement(name = "Customer") @Document public class Customer {@Idprivate Long id;@Fieldprivate String firstName;@Fieldprivate String lastName;@Fieldprivate LocalDate birthdate; }

CustomerConverter — 我們已經實現了Converter接口。此類用于Converter實現，并且負責將Java對象與文本數據進行編組。如果在處理期間發生異常，則應引發ConversionException。如果使用高級com.thoughtworks.xstream.XStream門面，則可以使用XStream.registerConverter（）方法注冊新的轉換器。

package com.example.config; import java.time.LocalDate; import java.time.format.DateTimeFormatter; import com.example.domain.Customer; import com.thoughtworks.xstream.converters.Converter; import com.thoughtworks.xstream.converters.MarshallingContext; import com.thoughtworks.xstream.converters.UnmarshallingContext; import com.thoughtworks.xstream.io.HierarchicalStreamReader; import com.thoughtworks.xstream.io.HierarchicalStreamWriter; public class CustomerConverter implements Converter {private static final DateTimeFormatter DT_FORMATTER = DateTimeFormatter.ofPattern("dd-MM-yyyy HH:mm:ss");@Overridepublic boolean canConvert(Class type) {return type.equals(Customer.class);}@Overridepublic void marshal(Object source, HierarchicalStreamWriter writer, MarshallingContext context) {// Don't do anything}@Overridepublic Object unmarshal(HierarchicalStreamReader reader, UnmarshallingContext context) {reader.moveDown();Customer customer = new Customer();customer.setId(Long.valueOf(reader.getValue()));reader.moveUp();reader.moveDown();customer.setFirstName(reader.getValue());reader.moveUp();reader.moveDown();customer.setLastName(reader.getValue());reader.moveUp();reader.moveDown();customer.setBirthdate(LocalDate.parse(reader.getValue(), DT_FORMATTER));return customer;} }

JobConfiguration- 這是負責執行批處理作業的主要類。在此類中，我們使用了各種Bean來執行單獨的任務。

StaxEventItemReader — 用于讀取基于StAX的XML輸入的項目讀取器。它從輸入的XML文檔中提取片段，該片段對應于要處理的記錄。片段用StartDocument和EndDocument事件包裝，以便可以像獨立XML文檔一樣對片段進行進一步處理。該實現不是線程安全的。

CustomerWriter —這是一個自定義類，可將數據寫入MongoDB。

step1 —此步驟配置ItemReader和ItemWriter，但是ItemProcessor是可選步驟，我們已跳過。

作業- 代表作業的批處理域對象。Job是一個顯式抽象，表示開發人員指定的作業配置。應當注意，重新啟動策略是整體上應用的，而不是步驟。

package com.example.config; import java.util.HashMap; import java.util.Map; import org.springframework.batch.core.Job; import org.springframework.batch.core.Step; import org.springframework.batch.core.configuration.annotation.JobBuilderFactory; import org.springframework.batch.core.configuration.annotation.StepBuilderFactory; import org.springframework.batch.item.xml.StaxEventItemReader; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.core.io.ClassPathResource; import org.springframework.oxm.xstream.XStreamMarshaller; import com.example.domain.Customer; import com.example.writer.CustomerWriter; @Configuration public class JobConfiguration {@Autowiredprivate JobBuilderFactory jobBuilderFactory;@Autowiredprivate StepBuilderFactory stepBuilderFactory;@Beanpublic StaxEventItemReader<Customer> customerItemReader(){Map<String, Class> aliases = new HashMap<>();aliases.put("customer", Customer.class);CustomerConverter converter = new CustomerConverter();XStreamMarshaller ummarshaller = new XStreamMarshaller();ummarshaller.setAliases(aliases);ummarshaller.setConverters(converter);StaxEventItemReader<Customer> reader = new StaxEventItemReader<>();reader.setResource(new ClassPathResource("/data/customer.xml"));reader.setFragmentRootElementName("customer")reader.setUnmarshaller(ummarshaller);return reader;}@Beanpublic CustomerWriter customerWriter() {return new CustomerWriter();}@Beanpublic Step step1() throws Exception {return stepBuilderFactory.get("step1").<Customer, Customer>chunk(200).reader(customerItemReader()).writer(customerWriter()).build();}@Beanpublic Job job() throws Exception {return jobBuilderFactory.get("job").start(step1()).build();} }

application.properties

spring.data.mongodb.host=localhost

spring.data.mongodb.port=27017

Customer.xml —這是Spring Batch讀取的示例數據。

<?xml version="1.0" encoding="UTF-8" ?> <customers><customer><id>1</id><firstName>John</firstName><lastName>Doe</lastName><birthdate>10-10-1988 19:43:23</birthdate></customer><customer><id>2</id><firstName>James</firstName><lastName>Moss</lastName><birthdate>01-04-1991 10:20:23</birthdate></customer><customer><id>3</id><firstName>Jonie</firstName><lastName>Gamble</lastName><birthdate>21-07-1982 11:12:13</birthdate></customer><customer><id>4</id><firstName>Mary</firstName><lastName>Kline</lastName><birthdate>07-08-1973 11:27:42</birthdate></customer><customer><id>5</id><firstName>William</firstName><lastName>Lockhart</lastName><birthdate>04-04-1994 04:15:11</birthdate></customer><customer><id>6</id><firstName>John</firstName><lastName>Doe</lastName><birthdate>10-10-1988 19:43:23</birthdate></customer><customer><id>7</id><firstName>Kristi</firstName><lastName>Dukes</lastName><birthdate>17-09-1983 21:22:23</birthdate></customer><customer><id>8</id><firstName>Angel</firstName><lastName>Porter</lastName><birthdate>15-12-1980 18:09:09</birthdate></customer><customer><id>9</id><firstName>Mary</firstName><lastName>Johnston</lastName><birthdate>07-07-1987 19:43:03</birthdate></customer><customer><id>10</id><firstName>Linda</firstName><lastName>Rodriguez</lastName><birthdate>16-09-1991 09:13:43</birthdate></customer><customer><id>11</id><firstName>Phillip</firstName><lastName>Lopez</lastName><birthdate>18-12-1965 11:10:09</birthdate></customer><customer><id>12</id><firstName>Peter</firstName><lastName>Dixon</lastName><birthdate>09-05-1996 19:09:23</birthdate></customer> </customers>

MainApp — SpringBatchMongodbApplication可以作為Spring Boot項目運行。

package com.example;

import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing; import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; import org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration; import org.springframework.data.mongodb.repository.config.EnableMongoRepositories; @SpringBootApplication(exclude = {DataSourceAutoConfiguration.class}) @EnableBatchProcessing @EnableMongoRepositories(basePackages = "com.example.repository") public class SpringBatchMongodbApplication {public static void main(String[] args) {SpringApplication.run(SpringBatchMongodbApplication.class, args);} }

輸出：我們可以得出結論，Spring Batch已經使用建議的模式/文檔類型讀取了數據并將其寫入MongoDB。

db.getCollection('customer').find({}) /* 1 */ {"_id" : NumberLong(1),"firstName" : "John","lastName" : "Doe","birthdate" : ISODate("1988-10-09T18:30:00.000Z"),"_class" : "com.example.domain.Customer" } /* 2 */ {"_id" : NumberLong(2),"firstName" : "James","lastName" : "Moss","birthdate" : ISODate("1991-03-31T18:30:00.000Z"),"_class" : "com.example.domain.Customer" } /* 3 */ {"_id" : NumberLong(3),"firstName" : "Jonie","lastName" : "Gamble","birthdate" : ISODate("1982-07-20T18:30:00.000Z"),"_class" : "com.example.domain.Customer" } /* 4 */ {"_id" : NumberLong(4),"firstName" : "Mary","lastName" : "Kline","birthdate" : ISODate("1973-08-06T18:30:00.000Z"),"_class" : "com.example.domain.Customer" } /* 5 */ {"_id" : NumberLong(5),"firstName" : "William","lastName" : "Lockhart","birthdate" : ISODate("1994-04-03T18:30:00.000Z"),"_class" : "com.example.domain.Customer" } /* 6 */ {"_id" : NumberLong(6),"firstName" : "John","lastName" : "Doe","birthdate" : ISODate("1988-10-09T18:30:00.000Z"),"_class" : "com.example.domain.Customer" } /* 7 */ {"_id" : NumberLong(7),"firstName" : "Kristi","lastName" : "Dukes","birthdate" : ISODate("1983-09-16T18:30:00.000Z"),"_class" : "com.example.domain.Customer" } /* 8 */ {"_id" : NumberLong(8),"firstName" : "Angel","lastName" : "Porter","birthdate" : ISODate("1980-12-14T18:30:00.000Z"),"_class" : "com.example.domain.Customer" } /* 9 */ {"_id" : NumberLong(9),"firstName" : "Mary","lastName" : "Johnston","birthdate" : ISODate("1987-07-06T18:30:00.000Z"),"_class" : "com.example.domain.Customer" } /* 10 */ {"_id" : NumberLong(10),"firstName" : "Linda","lastName" : "Rodriguez","birthdate" : ISODate("1991-09-15T18:30:00.000Z"),"_class" : "com.example.domain.Customer" } /* 11 */ {"_id" : NumberLong(11),"firstName" : "Phillip","lastName" : "Lopez","birthdate" : ISODate("1965-12-17T18:30:00.000Z"),"_class" : "com.example.domain.Customer" } /* 12 */ {"_id" : NumberLong(12),"firstName" : "Peter","lastName" : "Dixon","birthdate" : ISODate("1996-05-08T18:30:00.000Z"),"_class" : "com.example.domain.Customer" }

> 喜歡這篇文章的可以點個贊，歡迎大家留言評論，記得關注我，每天持續更新技術干貨、職場趣事、海量面試資料等等

> 如果你對java技術很感興趣也可以交流學習，共同學習進步。

> 不要再用"沒有時間“來掩飾自己思想上的懶惰！趁年輕，使勁拼，給未來的自己一個交代

文章寫道這里，歡迎完善交流。最后奉上近期整理出來的一套完整的java架構思維導圖，分享給大家對照知識點參考學習。有更多JVM、Mysql、Tomcat、Spring Boot、Spring Cloud、Zookeeper、Kafka、RabbitMQ、RockerMQ、Redis、ELK、Git等Java干貨

創作挑戰賽新人創作獎勵來咯，堅持創作打卡瓜分現金大獎

總結

以上是生活随笔為你收集整理的java mongo 获取所有数据库_Spring Batch —从XML读取并写入Mongo的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。