edu.umass.cs.reconfiguration

Class Reconfigurator<NodeIDType>

java.lang.Object

edu.umass.cs.reconfiguration.Reconfigurator<NodeIDType>

Type Parameters:

NodeIDType - This class is the main Reconfigurator module. It issues reconfiguration commands to ActiveReplicas and also responds to client requests to create or delete names or request the list of active replicas for a name. It relies on the following helper protocol tasks: WaitAckStopEpoch, WaitAckStartEpoch, WaitAckDropEpoch, CommitWorker, WaitPrimaryExecution. The last one is to enable exactly one primary Reconfigurator in the common case to conduct reconfigurations but ensure that others safely complete the reconfiguration if the primary fails to do so. CommitWorker is a worker that is needed to ensure that a paxos-coordinated request eventually gets committed; we need this property to ensure that a reconfiguration operation terminates, but paxos itself provides us no such liveness guarantee. This class now supports add/remove operations for the set of Reconfigurator nodes. This is somewhat tricky, but works correctly. A detailed, formal description of the protocol is TBD. The documentation further below in this class explains the main ideas.

All Implemented Interfaces:

ExecutedCallback, PacketDemultiplexer<Request>, ReconfiguratorCallback, ReconfiguratorFunctions

public class Reconfigurator<NodeIDType>
extends java.lang.Object
implements PacketDemultiplexer<Request>, ReconfiguratorCallback, ReconfiguratorFunctions

Author:

V. Arun

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Reconfigurator.ProfilerKeys For profiling statistics in DelayProfiler.

Field Summary

Fields

Modifier and Type	Field and Description
protected ReconfiguratorProtocolTask<NodeIDType>	protocolTask

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	Reconfigurator(ReconfigurableNodeConfig<NodeIDType> nc, SSLMessenger<NodeIDType,org.json.JSONObject> m, boolean startCleanSlate)

Method Summary

Modifier and Type	Method and Description
void	close() Close gracefully.
void	executed(Request request, boolean handled) Reconfiguration is initiated using a callback because the intent to conduct a reconfiguration must be persistently committed before initiating the reconfiguration.
static int	getClientFacingClearPort(int port)
static int	getClientFacingSSLPort(int port) Refer ActiveReplica.getClientFacingSSLPort(int).
Callback<Request,ReconfiguratorRequest>	getDefaultCallback()
java.util.Set<ReconfigurationPacket.PacketType>	getPacketTypes()
protected ActiveReplica<NodeIDType>	getReconfigurableReconfiguratorAsActiveReplica() This treats the reconfigurator itself as an "active replica" in order to be able to reconfigure reconfigurator groups.
static java.lang.String	getTaskKey(java.lang.Class<?> C, BasicReconfigurationPacket<?> rcPacket, java.lang.String myID)
protected static java.lang.String	getTaskKeyPrev(java.lang.Class<?> C, BasicReconfigurationPacket<?> rcPacket, java.lang.String myID)
GenericMessagingTask<NodeIDType,?>[]	handleCreateServiceName(CreateServiceName create, ProtocolTask<NodeIDType,ReconfigurationPacket.PacketType,java.lang.String>[] ptasks) Create service name is a special case of reconfiguration where the previous group is non-existent.
GenericMessagingTask<NodeIDType,?>[]	handleCreateServiceName(CreateServiceName create, ProtocolTask<NodeIDType,ReconfigurationPacket.PacketType,java.lang.String>[] ptasks, Callback<Request,ReconfiguratorRequest> callback)
GenericMessagingTask<NodeIDType,?>[]	handleDeleteServiceName(DeleteServiceName delete, ProtocolTask<NodeIDType,ReconfigurationPacket.PacketType,java.lang.String>[] ptasks) We need to ensure that both the stop/drop at actives happens atomically with the removal of the record at reconfigurators.
GenericMessagingTask<NodeIDType,?>[]	handleDeleteServiceName(DeleteServiceName delete, ProtocolTask<NodeIDType,ReconfigurationPacket.PacketType,java.lang.String>[] ptasks, Callback<Request,ReconfiguratorRequest> callback)
GenericMessagingTask<NodeIDType,?>[]	handleDemandReport(DemandReport<NodeIDType> report, ProtocolTask<NodeIDType,ReconfigurationPacket.PacketType,java.lang.String>[] ptasks) Incorporates demand reports (possibly but not necessarily with replica coordination), checks for reconfiguration triggers, and initiates reconfiguration if needed.
GenericMessagingTask<NodeIDType,?>[]	handleEchoRequest(EchoRequest echo, ProtocolTask<NodeIDType,ReconfigurationPacket.PacketType,java.lang.String>[] ptasks)
boolean	handleMessage(Request incoming, NIOHeader header)
GenericMessagingTask<NodeIDType,?>[]	handleRCRecordRequest(RCRecordRequest<NodeIDType> rcRecReq, ProtocolTask<NodeIDType,ReconfigurationPacket.PacketType,java.lang.String>[] ptasks) Simply hand over DB request to DB.
GenericMessagingTask<?,?>[]	handleReconfigureActiveNodeConfig(ReconfigureActiveNodeConfig<NodeIDType> changeActives, ProtocolTask<NodeIDType,ReconfigurationPacket.PacketType,java.lang.String>[] ptasks)
GenericMessagingTask<?,?>[]	handleReconfigureRCNodeConfig(ReconfigureRCNodeConfig<NodeIDType> changeRC, ProtocolTask<NodeIDType,ReconfigurationPacket.PacketType,java.lang.String>[] ptasks) Handles a request to add or delete a reconfigurator from the set of all reconfigurators in NodeConfig.
GenericMessagingTask<NodeIDType,?>[]	handleRequestActiveReplicas(RequestActiveReplicas request, ProtocolTask<NodeIDType,ReconfigurationPacket.PacketType,java.lang.String>[] ptasks) This method simply looks up and returns the current set of active replicas.
GenericMessagingTask<NodeIDType,?>[]	handleRequestActiveReplicas(RequestActiveReplicas request, ProtocolTask<NodeIDType,ReconfigurationPacket.PacketType,java.lang.String>[] ptasks, Callback<Request,ReconfiguratorRequest> callback)
boolean	preExecuted(Request request)
ReconfiguratorRequest	sendRequest(ReconfiguratorRequest request)
RequestCallbackFuture<ReconfiguratorRequest>	sendRequest(ReconfiguratorRequest request, Callback<Request,ReconfiguratorRequest> callback)
java.lang.String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface edu.umass.cs.reconfiguration.interfaces.ReconfiguratorCallback

preCheckpoint, preRestore

Field Detail

protocolTask

protected final ReconfiguratorProtocolTask<NodeIDType> protocolTask

Constructor Detail

Reconfigurator

protected Reconfigurator(ReconfigurableNodeConfig<NodeIDType> nc,
                         SSLMessenger<NodeIDType,org.json.JSONObject> m,
                         boolean startCleanSlate)
                  throws java.io.IOException

Throws:

java.io.IOException

Method Detail

getReconfigurableReconfiguratorAsActiveReplica

protected ActiveReplica<NodeIDType> getReconfigurableReconfiguratorAsActiveReplica()

This treats the reconfigurator itself as an "active replica" in order to be able to reconfigure reconfigurator groups.

handleMessage

public boolean handleMessage(Request incoming,
                             NIOHeader header)

Specified by:

handleMessage in interface PacketDemultiplexer<Request>

Returns:

The return value should return true if the handler handled the message and doesn't want any other BasicPacketDemultiplexer to handle the message.

getPacketTypes

public java.util.Set<ReconfigurationPacket.PacketType> getPacketTypes()

Returns:

Packet types handled by Reconfigurator. Refer ReconfigurationPacket.

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object

close

public void close()

Close gracefully.

handleDemandReport

public GenericMessagingTask<NodeIDType,?>[] handleDemandReport(DemandReport<NodeIDType> report,
                                                               ProtocolTask<NodeIDType,ReconfigurationPacket.PacketType,java.lang.String>[] ptasks)

Incorporates demand reports (possibly but not necessarily with replica coordination), checks for reconfiguration triggers, and initiates reconfiguration if needed.

Parameters:

report -

ptasks -

Returns:

MessagingTask, typically null. No protocol tasks spawned.

handleCreateServiceName

public GenericMessagingTask<NodeIDType,?>[] handleCreateServiceName(CreateServiceName create,
                                                                    ProtocolTask<NodeIDType,ReconfigurationPacket.PacketType,java.lang.String>[] ptasks)

Create service name is a special case of reconfiguration where the previous group is non-existent.

Parameters:

create -

ptasks -

Returns:

Messaging task, typically null. No protocol tasks spawned.

handleCreateServiceName

public GenericMessagingTask<NodeIDType,?>[] handleCreateServiceName(CreateServiceName create,
                                                                    ProtocolTask<NodeIDType,ReconfigurationPacket.PacketType,java.lang.String>[] ptasks,
                                                                    Callback<Request,ReconfiguratorRequest> callback)

Parameters:

create -

ptasks -

callback -

Returns:

Messaging task, typically null. No protocol tasks spawned.

handleRCRecordRequest

public GenericMessagingTask<NodeIDType,?>[] handleRCRecordRequest(RCRecordRequest<NodeIDType> rcRecReq,
                                                                  ProtocolTask<NodeIDType,ReconfigurationPacket.PacketType,java.lang.String>[] ptasks)

Simply hand over DB request to DB. The only type of RC record that can come here is one announcing reconfiguration completion. Reconfiguration initiation messages are derived locally and coordinated through paxos, not received from outside.

Parameters:

rcRecReq -

ptasks -

Returns:

Messaging task, typically null unless TWO_PAXOS_RC.

handleDeleteServiceName

public GenericMessagingTask<NodeIDType,?>[] handleDeleteServiceName(DeleteServiceName delete,
                                                                    ProtocolTask<NodeIDType,ReconfigurationPacket.PacketType,java.lang.String>[] ptasks)

We need to ensure that both the stop/drop at actives happens atomically with the removal of the record at reconfigurators. To accomplish this, we first mark the record as stopped at reconfigurators, then wait for the stop/drop tasks to finish, and finally coordinate the completion notification so that reconfigurators can completely remove the record from their DB.

Parameters:

delete -

ptasks -

Returns:

Messaging task, typically null. No protocol tasks spawned.

handleDeleteServiceName

public GenericMessagingTask<NodeIDType,?>[] handleDeleteServiceName(DeleteServiceName delete,
                                                                    ProtocolTask<NodeIDType,ReconfigurationPacket.PacketType,java.lang.String>[] ptasks,
                                                                    Callback<Request,ReconfiguratorRequest> callback)

Parameters:

delete -

ptasks -

callback -

Returns:

Messaging task, typically null. No protocol tasks spawned.

getDefaultCallback

public Callback<Request,ReconfiguratorRequest> getDefaultCallback()

Returns:

Default callback for auto-invoked methods. Meant only for internal use.

handleRequestActiveReplicas

public GenericMessagingTask<NodeIDType,?>[] handleRequestActiveReplicas(RequestActiveReplicas request,
                                                                        ProtocolTask<NodeIDType,ReconfigurationPacket.PacketType,java.lang.String>[] ptasks)

This method simply looks up and returns the current set of active replicas. Maintaining this state consistently is the primary and only existential purpose of reconfigurators.

Parameters:

request -

ptasks -

Returns:

Messaging task returning the set of active replicas to the requestor. No protocol tasks spawned.

handleRequestActiveReplicas

public GenericMessagingTask<NodeIDType,?>[] handleRequestActiveReplicas(RequestActiveReplicas request,
                                                                        ProtocolTask<NodeIDType,ReconfigurationPacket.PacketType,java.lang.String>[] ptasks,
                                                                        Callback<Request,ReconfiguratorRequest> callback)

Parameters:

request -

ptasks -

callback -

Returns:

Messaging task returning the set of active replicas to the requestor. No protocol tasks spawned.

handleReconfigureRCNodeConfig

public GenericMessagingTask<?,?>[] handleReconfigureRCNodeConfig(ReconfigureRCNodeConfig<NodeIDType> changeRC,
                                                                 ProtocolTask<NodeIDType,ReconfigurationPacket.PacketType,java.lang.String>[] ptasks)

Handles a request to add or delete a reconfigurator from the set of all reconfigurators in NodeConfig. The reconfiguration record corresponding to NodeConfig is stored in the RC records table and the "active replica state" or the NodeConfig info itself is stored in a separate NodeConfig table in the DB.

Parameters:

changeRC -

ptasks -

Returns:

Messaging task typically null. No protocol tasks spawned.

handleReconfigureActiveNodeConfig

public GenericMessagingTask<?,?>[] handleReconfigureActiveNodeConfig(ReconfigureActiveNodeConfig<NodeIDType> changeActives,
                                                                     ProtocolTask<NodeIDType,ReconfigurationPacket.PacketType,java.lang.String>[] ptasks)

Parameters:

changeActives -

ptasks -

Returns:

Messaging task if any.

executed

public void executed(Request request,
                     boolean handled)

Reconfiguration is initiated using a callback because the intent to conduct a reconfiguration must be persistently committed before initiating the reconfiguration. Otherwise, the failure of say the initiating reconfigurator can leave an active replica group stopped indefinitely. Exactly one reconfigurator, the one that proposes the request initiating reconfiguration registers the callback. This initiating reconfigurator will spawn a WaitAckStopEpoch task when the initiating request is locally executed. The other replicas only spawn a WaitPrimaryExecution task as a double check that the initiating reconfigurator does complete the reconfiguration; if it does not, they will follow up with their own attempt after a timeout. This works because all three steps: WaitAckStopEpoch, WaitAckStartEpoch, and WaitAckDropEpoch are idempotent operations. A reconfiguration attempt can still get stuck if all reconfigurators crash or the only reconfigurators that committed the intent crash. So, a replica recovery procedure should ensure that replicas eventually execute committed but unexecuted requests. This naturally happens with paxos.

Specified by:

executed in interface ExecutedCallback

preExecuted

public boolean preExecuted(Request request)

Specified by:

preExecuted in interface ReconfiguratorCallback

Returns:

True if execution complete.

getClientFacingSSLPort

public static final int getClientFacingSSLPort(int port)

Refer ActiveReplica.getClientFacingSSLPort(int).

Parameters:

port -

Returns:

The client facing ssl port.

getClientFacingClearPort

public static final int getClientFacingClearPort(int port)

Parameters:

port -

Returns:

The client facing clear port.

getTaskKey

public static final java.lang.String getTaskKey(java.lang.Class<?> C,
                                                BasicReconfigurationPacket<?> rcPacket,
                                                java.lang.String myID)

Parameters:

C -

rcPacket -

myID -

Returns:

The task key.

getTaskKeyPrev

protected static java.lang.String getTaskKeyPrev(java.lang.Class<?> C,
                                                 BasicReconfigurationPacket<?> rcPacket,
                                                 java.lang.String myID)

handleEchoRequest

public GenericMessagingTask<NodeIDType,?>[] handleEchoRequest(EchoRequest echo,
                                                              ProtocolTask<NodeIDType,ReconfigurationPacket.PacketType,java.lang.String>[] ptasks)

Parameters:

echo -

ptasks -

Returns:

null

sendRequest

public ReconfiguratorRequest sendRequest(ReconfiguratorRequest request)

Specified by:

sendRequest in interface ReconfiguratorFunctions

Returns:

The response to request that will have the same type as the request.

sendRequest

public RequestCallbackFuture<ReconfiguratorRequest> sendRequest(ReconfiguratorRequest request,
                                                                Callback<Request,ReconfiguratorRequest> callback)

Specified by:

sendRequest in interface ReconfiguratorFunctions

Returns:

A RequestCallbackFuture object that can be used by the caller to retrieve the processed response.