Pub-Sub tries to deserialize messages from not subscribed topic

[AqlaSolutions]

We have a few nodes and some of them were using RunAsClient but now I disabled it and things went wrong. One of nodes (which originally didn't use RunAsClient) leverages pub-sub for internal communication between its actors. Somehow those internal messages arrive to another node and cause a deserialization exception (because another node doesn't know anything about those internal types). This happens even though that node doesn't subscribe to the topic where this message is published (but subscribes to another). It didn't happen before the RunAsClient change.

[23:11:14.902 ERR] Proto.Remote.RemoteMessageHandler ["localhost:7752"] Unable to deserialize message with "cluster.pubsub.PubSubBatchTransport"
System.Exception: Type with the specified name [redacted] not found
   at Proto.Remote.JsonSerializer.Deserialize(ByteString bytes, String typeName)
   at Proto.Remote.Serialization.Deserialize(String typeName, ByteString bytes, Int32 serializerId)
   at Proto.Cluster.PubSub.PubSubBatchTransport.<>c__DisplayClass0_0.<Deserialize>b__0(PubSubEnvelope e)
   at System.Linq.Enumerable.SelectIListIterator`2.ToList()
   at Proto.Cluster.PubSub.PubSubBatchTransport.Deserialize(ActorSystem system)
   at Proto.Remote.RemoteMessageHandler.HandleRemoteMessage(RemoteMessage currentMessage, String remoteAddress)   

Is this a bug or expected behavior?

[AqlaSolutions]

It looks like TopicActor starts at another non-publisher node so it has to deal with serialization. How can we force it to start at the local node for specific topics?

[rogeralsing]

Hi,
currently this is only one type of topic-actor, and that is cluster-wide so to say. it can run on any cluster node.
This is why you are seeing this behavior.

That being said.
Pub-Sub is still experimental. we could def expand the API to allow for more customization here.
e.g. currently the topic-actor kind is autoregistered for all cluster nodes.
We could make it so that you can register various topic types, with different configurations. and for different nodes that is.

That change would be pretty easy to solve. it´s mostly about allowing to have a non hardcoded kind for the topic actor.

Up for suggestions here