public class JdbcStreams
extends java.lang.Object
JdbcStreams
is a streams connector to a database via the
JDBC API java.sql
package.
The connector provides general SQL access to a database, enabling writing of a stream's tuples to a database, creating a stream from database query results, and other operations. Knowledge of the JDBC API is required.
Use of the connector involves:
DataSource
Connection
from the DataSource
executeStatement()
methods:
StatementSupplier
.
A StatementSupplier
creates a JDBC PreparedStatement
for an SQL statement
(e.g., a query, insert, update, etc operation).ParameterSetter
. A ParameterSetter
sets the parameter values in a generic PreparedStatement
.ResultsHandler
as required.
A ResultsHandler
processes a JDBC
ResultSet
created by executing a SQL statement,
optionally creating one or more tuples from the results
and adding them to a stream.Sample use:
// construct a connector to the database
JdbcStreams mydb = new JdbcStreams(
// fn to create the javax.sql.DataSource to the db
() -> {
Context ctx = new javax.naming.InitialContext();
return (DataSource) ctx.lookup("jdbc/myDb");
},
// fn to connect to the db (via the DataSource)
(dataSource,cn) -> dataSource.getConnection(username,pw)
);
// ----------------------------------------------------
//
// Write a Person stream to a table
//
TStream<Person> persons = ...
TSink sink = mydb.executeStatement(persons,
() -> "INSERT INTO persons VALUES(?,?,?)",
(person,stmt) -> {
stmt.setInt(1, person.getId());
stmt.setString(2, person.getFirstName());
stmt.setString(3, person.getLastName());
},
);
// ----------------------------------------------------
//
// Create a stream of Person from a PersonId tuple
//
TStream<PersonId> personIds = ...
TStream<Person> persons = mydb.executeStatement(personIds,
() -> "SELECT id, firstname, lastname FROM persons WHERE id = ?",
(personId,stmt) -> stmt.setInt(1,personId.getId()),
(personId,rs,exc,consumer) -> {
if (exc != null) {
// statement failed, do something
int ecode = exc.getErrorCode();
String state = exc.getSQLState();
... // consumer.accept(...) if desired.
}
else {
rs.next();
int id = resultSet.getInt("id");
String firstName = resultSet.getString("firstname");
String lastName = resultSet.getString("lastname");
consumer.accept(new Person(id, firstName, lastName));
}
}
);
persons.print();
// ----------------------------------------------------
//
// Delete all the rows from a table
//
TStream<String> beacon = topology.strings("once");
mydb.executeStatement(beacon,
() -> "DELETE FROM persons",
(tuple,stmt) -> { } // no params to set
);
Constructor and Description |
---|
JdbcStreams(Topology topology,
CheckedSupplier<javax.sql.DataSource> dataSourceFn,
CheckedFunction<javax.sql.DataSource,java.sql.Connection> connFn)
Create a connector that uses a JDBC
DataSource object to get
a database connection. |
Modifier and Type | Method and Description |
---|---|
<T> TSink<T> |
executeStatement(TStream<T> stream,
StatementSupplier stmtSupplier,
ParameterSetter<T> paramSetter)
For each tuple on
stream execute an SQL statement. |
<T,R> TStream<R> |
executeStatement(TStream<T> stream,
StatementSupplier stmtSupplier,
ParameterSetter<T> paramSetter,
ResultsHandler<T,R> resultsHandler)
For each tuple on
stream execute an SQL statement and
add 0 or more resulting tuples to a result stream. |
<T> TSink<T> |
executeStatement(TStream<T> stream,
Supplier<java.lang.String> stmtSupplier,
ParameterSetter<T> paramSetter)
For each tuple on
stream execute an SQL statement. |
<T,R> TStream<R> |
executeStatement(TStream<T> stream,
Supplier<java.lang.String> stmtSupplier,
ParameterSetter<T> paramSetter,
ResultsHandler<T,R> resultsHandler)
For each tuple on
stream execute an SQL statement and
add 0 or more resulting tuples to a result stream. |
public JdbcStreams(Topology topology, CheckedSupplier<javax.sql.DataSource> dataSourceFn, CheckedFunction<javax.sql.DataSource,java.sql.Connection> connFn)
DataSource
object to get
a database connection.
In some environments it's common for JDBC DataSource objects to have been registered in JNDI. In such cases the dataSourceFn can be:
() -> { Context ctx = new javax.naming.InitialContext();
return (DataSource) ctx.lookup("jdbc/" + logicalDbName);
}
Alternatively, a DataSource can be created using a dbms implementation's DataSource class. For example:
() -> { EmbeddedDataSource ds = new org.apache.derby.jdbc.EmbeddedDataSource();
ds.setDatabaseName(dbName);
ds.setCreateDatabase("create");
return ds;
}
Once dataSourceFn
returns a DataSource it will not be called again.
connFn
is called only if a new JDBC connection is needed.
It is not called per-processed-tuple. JDBC failures in
executeStatement()
can result in a JDBC connection getting
closed and connFn
is subsequently called to reconnect.
topology
- topology that this connector is fordataSourceFn
- function that yields the DataSource
for the database.connFn
- function that yields a Connection
from a DataSource
.public <T,R> TStream<R> executeStatement(TStream<T> stream, Supplier<java.lang.String> stmtSupplier, ParameterSetter<T> paramSetter, ResultsHandler<T,R> resultsHandler)
stream
execute an SQL statement and
add 0 or more resulting tuples to a result stream.
Same as using executeStatement(TStream, StatementSupplier, ParameterSetter, ResultsHandler)
specifying dataSource -> dataSource.prepareStatement(stmtSupplier.get()
}
for the StatementSupplier
.
T
- Tuple type for input streamR
- Tuple type of result streamstream
- tuples to execute a SQL statement on behalf ofstmtSupplier
- an SQL statementparamSetter
- function to set SQL statement parametersresultsHandler
- SQL ResultSet handlerpublic <T,R> TStream<R> executeStatement(TStream<T> stream, StatementSupplier stmtSupplier, ParameterSetter<T> paramSetter, ResultsHandler<T,R> resultsHandler)
stream
execute an SQL statement and
add 0 or more resulting tuples to a result stream.
Use to transform T tuples to R tuples, or enrich/update T tuples with additional information from a database. It can also be used to load a table into stream, using a T to trigger that. Or to execute non-ResultSet generating SQL statements and receive failure info and/or generate tuple(s) upon completion.
stmtSupplier
is called only once per new JDBC connection/reconnect.
It is not called per-tuple. Hence, with the exception of statement
parameters, the returned statement is expected to be unchanging.
Failures executing a statement can result in the connection getting
closed and subsequently reconnected, resulting in another
stmtSupplier
call.
resultsHandler
is called for every tuple.
If resultsHandler
throws an Exception, it is called a
second time for the tuple with a non-null exception argument.
T
- Tuple type for input streamR
- Tuple type of result streamstream
- tuples to execute a SQL statement on behalf ofstmtSupplier
- an SQL statementparamSetter
- function to set SQL statement parametersresultsHandler
- SQL ResultSet handlerexecuteStatement(TStream, Supplier, ParameterSetter, ResultsHandler)
public <T> TSink<T> executeStatement(TStream<T> stream, Supplier<java.lang.String> stmtSupplier, ParameterSetter<T> paramSetter)
stream
execute an SQL statement.
Same as using executeStatement(TStream, StatementSupplier, ParameterSetter)
specifying dataSource -> dataSource.prepareStatement(stmtSupplier.get()
}
for the StatementSupplier
.
T
- Tuple typestream
- tuples to execute a SQL statement on behalf ofstmtSupplier
- an SQL statementparamSetter
- function to set SQL statement parameterspublic <T> TSink<T> executeStatement(TStream<T> stream, StatementSupplier stmtSupplier, ParameterSetter<T> paramSetter)
stream
execute an SQL statement.
Use to write a stream of T to a table. More generally, use a T as a trigger to execute some SQL statement that doesn't yield a ResultSet.
Use a non-sink form of executeStatement()
(forms
that take a ResultsHandler
), if you want to:
T
- Tuple typestream
- tuples to execute a SQL statement on behalf ofstmtSupplier
- an SQL statementparamSetter
- function to set SQL statement parametersexecuteStatement(TStream, Supplier, ParameterSetter)
Copyright © 2016 The Apache Software Foundation. All Rights Reserved - bbe71fa-20161201-1641