public class StreamSession extends java.lang.Object implements IEndpointStateChangeSubscriber
StreamSession
. A streaming
session has the following life-cycle:
1. Session Initialization
(a) A node (the initiator in the following) create a new StreamSession
,
initialize it init(StreamResultFuture)
, and then start it (start()
).
Starting a session causes a StreamInitMessage
to be sent.
(b) Upon reception of that StreamInitMessage
, the follower creates its own StreamSession
,
and initializes it if it still does not exist.
(c) After the initiator sends the StreamInitMessage
, it invokes
onInitializationComplete()
to start the streaming prepare phase.
2. Streaming preparation phase
(a) A PrepareSynMessage
is sent that includes a) what files/sections this node will stream to the follower
(stored locally in a StreamTransferTask
, one for each table) and b) what the follower needs to
stream back (stored locally in a StreamReceiveTask
, one for each table).
(b) Upon reception of the PrepareSynMessage
, the follower records which files/sections it will receive
and send back a PrepareSynAckMessage
, which contains a summary of the files/sections that will be sent to
the initiator.
(c) When the initiator receives the PrepareSynAckMessage
, it records which files/sections it will
receive, and then goes to it's Streaming phase (see next section). If the intiator is to receive files,
it sends a PrepareAckMessage
to the follower to indicate that it can start streaming to the initiator.
(d) (Optional) If the follower receives a PrepareAckMessage
, it enters it's Streaming phase.
3. Streaming phase
(a) The streaming phase is started at each node by calling startStreamingFiles(boolean)
.
This will send, sequentially on each outbound streaming connection (see NettyStreamingMessageSender
),
an OutgoingStreamMessage
for each stream in each of the StreamTransferTask
.
Each OutgoingStreamMessage
consists of a StreamMessageHeader
that contains metadata about
the stream, followed by the stream content itself. Once all the files for a StreamTransferTask
are sent,
the task is marked complete StreamTransferTask.complete(int)
.
(b) On the receiving side, the incoming data is written to disk, and once the stream is fully received,
it will be marked as complete (StreamReceiveTask.received(IncomingStream)
). When all streams
for the StreamReceiveTask
have been received, the data is added to the CFS (and 2ndary indexes/MV are built),
and the task is marked complete (taskCompleted(StreamReceiveTask)
).
(b) If during the streaming of a particular stream an error occurs on the receiving end of a stream
(it may be either the initiator or the follower), the node will send a SessionFailedMessage
to the sender and close the stream session.
(c) When all transfer and receive tasks for a session are complete, the session moves to the Completion phase
(maybeCompleted()
).
4. Completion phase
(a) When a node enters the completion phase, it sends a CompleteMessage
to the peer, and then enter the
StreamSession.State.WAIT_COMPLETE
state. If it has already received a CompleteMessage
from the peer, session is complete and is then closed (closeSession(State)
). Otherwise, the node
switch to the StreamSession.State.WAIT_COMPLETE
state and send a CompleteMessage
to the other side.
In brief, the message passing looks like this (I for initiator, F for follwer):
(session init)
I: StreamInitMessage
(session prepare)
I: PrepareSynMessage
F: PrepareSynAckMessage
I: PrepareAckMessage
(stream - this can happen in both directions)
I: OutgoingStreamMessage
F: ReceivedMessage
(completion)
I/F: CompleteMessage
All messages which derive from StreamMessage
are sent by the standard internode messaging
(via MessagingService
, while the actual files themselves are sent by a special
"streaming" connection type. See NettyStreamingMessageSender
for details. Because of the asynchronousModifier and Type | Class and Description |
---|---|
static class |
StreamSession.State |
Modifier and Type | Field and Description |
---|---|
InetAddressAndPort |
peer
Streaming endpoint.
|
protected java.util.Set<StreamRequest> |
requests |
protected java.util.concurrent.ConcurrentHashMap<TableId,StreamTransferTask> |
transfers |
Constructor and Description |
---|
StreamSession(StreamOperation streamOperation,
InetAddressAndPort peer,
StreamConnectionFactory factory,
int index,
java.util.UUID pendingRepair,
PreviewKind previewKind)
Create new streaming session with the peer.
|
StreamSession(StreamOperation streamOperation,
OutboundConnectionSettings template,
StreamConnectionFactory factory,
int index,
java.util.UUID pendingRepair,
PreviewKind previewKind)
Create new streaming session with the peer.
|
Modifier and Type | Method and Description |
---|---|
void |
addStreamRequest(java.lang.String keyspace,
RangesAtEndpoint fullRanges,
RangesAtEndpoint transientRanges,
java.util.Collection<java.lang.String> columnFamilies)
Request data fetch task to this session.
|
boolean |
attach(io.netty.channel.Channel channel) |
void |
beforeChange(InetAddressAndPort endpoint,
EndpointState currentState,
ApplicationState newStateKey,
VersionedValue newValue) |
void |
complete()
Check if session is completed on receiving
StreamMessage.Type.COMPLETE message. |
StreamReceiver |
getAggregator(TableId tableId) |
NettyStreamingMessageSender |
getMessageSender() |
int |
getNumRequests() |
int |
getNumTransfers() |
java.util.List<OutgoingStream> |
getOutgoingStreamsForRanges(RangesAtEndpoint replicas,
java.util.Collection<ColumnFamilyStore> stores,
java.util.UUID pendingRepair,
PreviewKind previewKind) |
java.util.UUID |
getPendingRepair() |
PreviewKind |
getPreviewKind() |
SessionInfo |
getSessionInfo() |
StreamOperation |
getStreamOperation() |
void |
init(StreamResultFuture streamResult)
Bind this session to report to specific
StreamResultFuture and
perform pre-streaming initialization. |
boolean |
isPreview() |
boolean |
isSuccess()
Return if this session completed successfully.
|
void |
messageReceived(StreamMessage message) |
void |
onAlive(InetAddressAndPort endpoint,
EndpointState state) |
void |
onChange(InetAddressAndPort endpoint,
ApplicationState state,
VersionedValue value) |
void |
onDead(InetAddressAndPort endpoint,
EndpointState state) |
java.util.concurrent.Future |
onError(java.lang.Throwable e)
Call back for handling exception during streaming.
|
void |
onInitializationComplete()
Call back when connection initialization is complete to start the prepare phase.
|
void |
onJoin(InetAddressAndPort endpoint,
EndpointState epState)
Use to inform interested parties about the change in the state
for specified endpoint
|
void |
onRemove(InetAddressAndPort endpoint) |
void |
onRestart(InetAddressAndPort endpoint,
EndpointState epState)
Called whenever a node is restarted.
|
java.util.UUID |
planId() |
void |
prepare(java.util.Collection<StreamRequest> requests,
java.util.Collection<StreamSummary> summaries)
Prepare this session for sending/receiving files.
|
void |
prepareReceiving(StreamSummary summary) |
void |
progress(java.lang.String filename,
ProgressInfo.Direction direction,
long bytes,
long total) |
void |
receive(IncomingStreamMessage message)
Call back after receiving a stream.
|
void |
received(TableId tableId,
int sequenceNumber) |
void |
sessionFailed()
Call back on receiving
StreamMessage.Type.SESSION_FAILED message. |
int |
sessionIndex() |
void |
start()
invoked by the node that begins the stream session (it may be sending files, receiving files, or both)
|
StreamSession.State |
state() |
void |
state(StreamSession.State newState)
Set current state to
newState . |
StreamOperation |
streamOperation() |
void |
streamSent(OutgoingStreamMessage message)
Call back after sending StreamMessageHeader.
|
void |
taskCompleted(StreamReceiveTask completedTask) |
void |
taskCompleted(StreamTransferTask completedTask) |
public final InetAddressAndPort peer
StreamSession
is identified by this InetAddressAndPort which is broadcast address of the node streaming.protected final java.util.Set<StreamRequest> requests
protected final java.util.concurrent.ConcurrentHashMap<TableId,StreamTransferTask> transfers
public StreamSession(StreamOperation streamOperation, InetAddressAndPort peer, StreamConnectionFactory factory, int index, java.util.UUID pendingRepair, PreviewKind previewKind)
public StreamSession(StreamOperation streamOperation, OutboundConnectionSettings template, StreamConnectionFactory factory, int index, java.util.UUID pendingRepair, PreviewKind previewKind)
public java.util.UUID planId()
public int sessionIndex()
public StreamOperation streamOperation()
public StreamOperation getStreamOperation()
public java.util.UUID getPendingRepair()
public boolean isPreview()
public PreviewKind getPreviewKind()
public StreamReceiver getAggregator(TableId tableId)
public void init(StreamResultFuture streamResult)
StreamResultFuture
and
perform pre-streaming initialization.streamResult
- result to report topublic boolean attach(io.netty.channel.Channel channel)
public void start()
public void addStreamRequest(java.lang.String keyspace, RangesAtEndpoint fullRanges, RangesAtEndpoint transientRanges, java.util.Collection<java.lang.String> columnFamilies)
keyspace
- Requesting keyspacefullRanges
- Ranges to retrieve data that will return full data from the sourcetransientRanges
- Ranges to retrieve data that will return transient data from the sourcecolumnFamilies
- ColumnFamily names. Can be empty if requesting all CF under the keyspace.public java.util.List<OutgoingStream> getOutgoingStreamsForRanges(RangesAtEndpoint replicas, java.util.Collection<ColumnFamilyStore> stores, java.util.UUID pendingRepair, PreviewKind previewKind)
public void state(StreamSession.State newState)
newState
.newState
- new state to setpublic StreamSession.State state()
public NettyStreamingMessageSender getMessageSender()
public boolean isSuccess()
public void messageReceived(StreamMessage message)
public void onInitializationComplete()
public java.util.concurrent.Future onError(java.lang.Throwable e)
public void prepare(java.util.Collection<StreamRequest> requests, java.util.Collection<StreamSummary> summaries)
public void streamSent(OutgoingStreamMessage message)
message
- sent stream messagepublic void receive(IncomingStreamMessage message)
message
- received streampublic void progress(java.lang.String filename, ProgressInfo.Direction direction, long bytes, long total)
public void received(TableId tableId, int sequenceNumber)
public void complete()
StreamMessage.Type.COMPLETE
message.public void sessionFailed()
StreamMessage.Type.SESSION_FAILED
message.public SessionInfo getSessionInfo()
public void taskCompleted(StreamReceiveTask completedTask)
public void taskCompleted(StreamTransferTask completedTask)
public void onJoin(InetAddressAndPort endpoint, EndpointState epState)
IEndpointStateChangeSubscriber
onJoin
in interface IEndpointStateChangeSubscriber
endpoint
- endpoint for which the state change occurred.epState
- state that actually changed for the above endpoint.public void beforeChange(InetAddressAndPort endpoint, EndpointState currentState, ApplicationState newStateKey, VersionedValue newValue)
beforeChange
in interface IEndpointStateChangeSubscriber
public void onChange(InetAddressAndPort endpoint, ApplicationState state, VersionedValue value)
onChange
in interface IEndpointStateChangeSubscriber
public void onAlive(InetAddressAndPort endpoint, EndpointState state)
onAlive
in interface IEndpointStateChangeSubscriber
public void onDead(InetAddressAndPort endpoint, EndpointState state)
onDead
in interface IEndpointStateChangeSubscriber
public void onRemove(InetAddressAndPort endpoint)
onRemove
in interface IEndpointStateChangeSubscriber
public void onRestart(InetAddressAndPort endpoint, EndpointState epState)
IEndpointStateChangeSubscriber
state.isAlive() == false
as state
is from before the restarted node is marked up.onRestart
in interface IEndpointStateChangeSubscriber
public void prepareReceiving(StreamSummary summary)
public int getNumRequests()
public int getNumTransfers()
Copyright © 2009-2019 The Apache Software Foundation