Extract Transform Load (ETL) ExampleThe ETL (Extract, Transform, Load) example shows how to load data into a database using Camel. In this example we will poll for files, transform them and then store them in the database via the JPA component. OverviewThis example lives in the examples/camel-example-etl directory and will poll for XML files in the child src/data directory. When the files are detected, they are converted, using the fallback JAXB The code for this example is as follows
The there is the spring configuration file in src/resources/META-INF/services/camel-context.xml Code walkthroughSo lets start with the route definition in EtlRoutes public class EtlRoutes extends SpringRouteBuilder { public void configure() throws Exception { from("file:src/data?noop=true").convertBodyTo(PersonDocument.class) // .intercept(transactionInterceptor()) .to("jpa:org.apache.camel.example.etl.CustomerEntity"); // the following will dump the database to files from("jpa:org.apache.camel.example.etl.CustomerEntity?consumeDelete=false?consumer.delay=3000&consumeLockEntity=false") .setHeader(FileComponent.HEADER_FILE_NAME, el("${in.body.userName}.xml")) .to("file:target/customers?append=false"); } } The above sets up a route from the src/data directory. Notice we're using the noop mode of the File component so that the files are not moved or deleted when they are processed (so when the tool is restarted they will be processed again). We're converting the body of the message to a PersonDocument Then we send the message with a PersonDocument body to the JPA endpoint. Notice how this endpoint specifies the expected type. So the Type Converter is gonna try convert the PersonDocument to a CustomerEntity @Converter public class CustomerTransformer { private static final transient Log LOG = LogFactory.getLog(CustomerTransformer.class); private JpaTemplate template; public CustomerTransformer(JpaTemplate template) { this.template = template; } /** * A transformation method to convert a person document into a customer * entity */ @Converter public CustomerEntity toCustomer(PersonDocument doc) { String user = doc.getUser(); CustomerEntity customer = findCustomerByName(user); // lets convert information from the document into the entity bean customer.setFirstName(doc.getFirstName()); customer.setSurname(doc.getLastName()); customer.setCity(doc.getCity()); LOG.debug("Created customer: " + customer); return customer; } /** * Finds a customer for the given username, or creates and inserts a new one */ protected CustomerEntity findCustomerByName(String user) { List<CustomerEntity> list = template.find("select x from " + CustomerEntity.class.getName() + " x where x.userName = ?1", user); if (list.isEmpty()) { CustomerEntity answer = new CustomerEntity(); answer.setUserName(user); template.persist(answer); return answer; } else { return list.get(0); } } } which performs the necessary conversion to an entity bean which is then stored in the database Running the exampleTo run the example we use the Camel Maven Plugin. For example from the source or binary distribution the following should work cd examples/camel-example-etl mvn camel:run If you prefer you can just run the Main directly using mvn compile exec:java Failing that you can run the Main from inside your IDE if you prefer. Follow the Building instructions to create an Eclipse/IDEA project to import |