Str vs unicode type of keyspace, migrations_table in config.py

[lgabo]

Hi, thanks to this tool we started to import our schema to our cassandra db. But we find out that in case of str type of keyspace and migrations_table the params in sql command are not apostrophed in cassandra driver and the migration fails with message "Keyspace metadata was not refreshed. See log for details." or "Table metadata was not refreshed. See log for details."

We use:
cassandra-driver (3.22.0)
cassandra-migrate (0.3.3)
Python 2.7.5

Logs:
cassandra-migrate -H cas2 -p 9042 -u cassandra -P cassandra reset
WARNING:cassandra.cluster:Cluster.init called with contact_points specified, but no load_balancing_policy. In the next major version, this will raise an error; please specify a load-balancing policy. (contact_points = [u'cas2'], lbp = None)
The reset operation cannot be undone. Are you sure? [y/N] y
INFO:Migrator:Dropping existing keyspace 'ods'
WARNING:cassandra.connection:An authentication challenge was not sent, this is suspicious because the driver expects authentication (configured authenticator = PlainTextAuthenticator)
INFO:cassandra.cluster:New Cassandra host <Host: 10.180.137.167:9042 SearchAnalytics> discovered
INFO:cassandra.cluster:New Cassandra host <Host: 10.180.137.166:9042 SearchAnalytics> discovered
WARNING:cassandra.connection:An authentication challenge was not sent, this is suspicious because the driver expects authentication (configured authenticator = PlainTextAuthenticator)
WARNING:cassandra.connection:An authentication challenge was not sent, this is suspicious because the driver expects authentication (configured authenticator = PlainTextAuthenticator)
WARNING:cassandra.connection:An authentication challenge was not sent, this is suspicious because the driver expects authentication (configured authenticator = PlainTextAuthenticator)
INFO:cassandra.metadata:2018,get_keyspace, type keyspace = '<type 'unicode'>'
INFO:cassandra.metadata:2019,get_keyspace, where_clause ' WHERE keyspace_name = 'ods''
INFO:Migrator:Creating keyspace 'ods'
INFO:cassandra.metadata:2018,get_keyspace, type keyspace = '<type 'unicode'>'
INFO:cassandra.metadata:2019,get_keyspace, where_clause ' WHERE keyspace_name = 'ods''
INFO:cassandra.metadata:2018,get_keyspace, type keyspace = '<type 'unicode'>'
INFO:cassandra.metadata:2019,get_keyspace, where_clause ' WHERE keyspace_name = 'ods''
INFO:Migrator:_ensure_keyspace 'ods'
INFO:Migrator:_ensure_keyspace type '<class 'future.types.newstr.newstr'>'
INFO:cassandra.metadata:2018,get_keyspace, type keyspace = '<type 'unicode'>'
INFO:cassandra.metadata:2019,get_keyspace, where_clause ' WHERE keyspace_name = 'ods''
(My comment: Here is the problem, the keyspace param comes here as str and then in the where_clause it is not single quoted, I assume)
INFO:cassandra.metadata:2018,get_keyspace, type keyspace = '<class 'future.types.newstr.newstr'>'
INFO:cassandra.metadata:2019,get_keyspace, where_clause ' WHERE keyspace_name = ods'
WARNING:cassandra.connection:An authentication challenge was not sent, this is suspicious because the driver expects authentication (configured authenticator = PlainTextAuthenticator)
Traceback (most recent call last):
File "/usr/bin/cassandra-migrate", line 18, in
main()
File "/usr/lib/python2.7/site-packages/cassandra_migrate/cli.py", line 149, in main
cmd_method(opts)
File "/usr/lib/python2.7/site-packages/cassandra_migrate/migrator.py", line 100, in wrapper
return func(self, opts, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/cassandra_migrate/migrator.py", line 558, in reset
self.migrate(opts)
File "/usr/lib/python2.7/site-packages/cassandra_migrate/migrator.py", line 100, in wrapper
return func(self, opts, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/cassandra_migrate/migrator.py", line 536, in migrate
self._ensure_keyspace()
File "/usr/lib/python2.7/site-packages/cassandra_migrate/migrator.py", line 259, in _ensure_keyspace
self.cluster.refresh_keyspace_metadata(self.config.keyspace)
File "cassandra/cluster.py", line 2155, in cassandra.cluster.Cluster.refresh_keyspace_metadata
cassandra.DriverException: Keyspace metadata was not refreshed. See log for details.

If I make the following changes (the keyspace and migrations_table converted to unicode) in config.py, then the migration passes.

cassandra-migrate/cassandra_migrate/config.py

Line 40 in 62df1d8

def _assert_type(data, key, tpe, default=None):

def _assert_type(data, key, tpe, default=None):
"""Extract and verify if a key in a dictionary has a given type"""
value = data.get(key, default)
if tpe == str:
value = str(value)
if tpe == unicode:
value = unicode(value)
if not isinstance(value, tpe):
raise ValueError("Config error: {}: expected {}, found {}".format(
key, tpe, type(value)))
return value

cassandra-migrate/cassandra_migrate/config.py

Line 77 in 62df1d8

self.keyspace = _assert_type(data, 'keyspace', str)

self.keyspace = _assert_type(data, 'keyspace', unicode)

cassandra-migrate/cassandra_migrate/config.py

Line 94 in 62df1d8

self.migrations_table = _assert_type(data, 'migrations_table', str,

self.migrations_table = _assert_type(data, 'migrations_table', unicode,
default='database_migrations')

Logs:
cassandra-migrate -H cas2 -p 9042 -u cassandra -P cassandra reset
WARNING:cassandra.cluster:Cluster.init called with contact_points specified, but no load_balancing_policy. In the next major version, this will raise an error; please specify a load-balancing policy. (contact_points = [u'cas2'], lbp = None)
The reset operation cannot be undone. Are you sure? [y/N] y
INFO:Migrator:Dropping existing keyspace 'ods'
WARNING:cassandra.connection:An authentication challenge was not sent, this is suspicious because the driver expects authentication (configured authenticator = PlainTextAuthenticator)
INFO:cassandra.cluster:New Cassandra host <Host: 10.180.137.167:9042 SearchAnalytics> discovered
INFO:cassandra.cluster:New Cassandra host <Host: 10.180.137.166:9042 SearchAnalytics> discovered
WARNING:cassandra.connection:An authentication challenge was not sent, this is suspicious because the driver expects authentication (configured authenticator = PlainTextAuthenticator)
WARNING:cassandra.connection:An authentication challenge was not sent, this is suspicious because the driver expects authentication (configured authenticator = PlainTextAuthenticator)
WARNING:cassandra.connection:An authentication challenge was not sent, this is suspicious because the driver expects authentication (configured authenticator = PlainTextAuthenticator)
INFO:cassandra.metadata:2018,get_keyspace, type keyspace = '<type 'unicode'>'
INFO:cassandra.metadata:2019,get_keyspace, where_clause ' WHERE keyspace_name = 'ods''
INFO:Migrator:Creating keyspace 'ods'
INFO:cassandra.metadata:2018,get_keyspace, type keyspace = '<type 'unicode'>'
INFO:cassandra.metadata:2019,get_keyspace, where_clause ' WHERE keyspace_name = 'ods''
INFO:cassandra.metadata:2018,get_keyspace, type keyspace = '<type 'unicode'>'
INFO:cassandra.metadata:2019,get_keyspace, where_clause ' WHERE keyspace_name = 'ods''
INFO:Migrator:_ensure_keyspace 'ods'
INFO:Migrator:_ensure_keyspace type '<type 'unicode'>'
INFO:cassandra.metadata:2018,get_keyspace, type keyspace = '<type 'unicode'>'
INFO:cassandra.metadata:2019,get_keyspace, where_clause ' WHERE keyspace_name = 'ods''
INFO:Migrator:Creating table 'database_migrations' in keyspace 'ods'
INFO:cassandra.metadata:2018,get_keyspace, type keyspace = '<type 'unicode'>'
INFO:cassandra.metadata:2019,get_keyspace, where_clause ' WHERE keyspace_name = 'ods''
INFO:Migrator:Pending migrations found. Current version: None, Latest version: 13
INFO:Migrator:Advancing to version 1
INFO:Migrator:Writing in-progress migration version 1: Migration("1.0.0-odsCassandraSchema.cql")
...

I assume this behavior is because Py2 vs Py3. Is this tool compatible with Py2 and Py3 too? Or how otherwise can be fixed this issue?
Thanks in advance & regards
Gabo.

[lgabo]

One more important note:
The original impl is working with PyYAML 3.13 pip package, but not with e.g. PyYAML 5.1.2 (I do not know the exact version from which it stopped working) and later. To work with later versions of PyYAML the above changes is necessary and also loader is needed to be defined for yaml.load() to fix following message:

YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.

cassandra-migrate/cassandra_migrate/config.py

Line 117 in 62df1d8

config = yaml.load(f)

        config = yaml.load(f,Loader=yaml.FullLoader)

Regards, Gabo.