About datahub 0.13.1 Ingestion nifi Lineage problem

[handscmjunghee]

datahub can't ingestion nifi 2.0.0-M2 ?

error info

packages/packaging/version.py", line 54, in parse
    return Version(version)
  File "/tmp/datahub/ingest/venv-nifi-b3b89d5496a000a4/lib/python3.10/site-packages/packaging/version.py", line 200, in __init__
    raise InvalidVersion(f"Invalid version: '{version}'")
packaging.version.InvalidVersion: Invalid version: '2.0.0-M2'

[lhoss]

We have the same issue, even before using Nifi 2.x , because of our custom nifi build version ('1.19.1-p2'), which also contains not just simple sem-ver numbers :
Would be great to have this fixed and allow (almost) any version strings

Full stacktrace , using Datahub 0.13.1 (CLI v 0.13.1.2):

  "cli": {
    "cli_version": "0.13.1.2",
    "cli_entry_location": "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/datahub/__init__.py",
...

[2024-05-14 13:47:57,852] ERROR    {datahub.entrypoints:201} - Command failed: Invalid version: '1.19.1-p2'
Traceback (most recent call last):
  File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/datahub/entrypoints.py", line 188, in main
    sys.exit(datahub(standalone_mode=False, **kwargs))
  File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/datahub/telemetry/telemetry.py", line 454, in wrapper
    raise e
  File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/datahub/telemetry/telemetry.py", line 403, in wrapper
    res = func(*args, **kwargs)
  File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/datahub/cli/ingest_cli.py", line 201, in run
    ret = loop.run_until_complete(run_ingestion_and_check_upgrade())
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
    return future.result()
  File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/datahub/cli/ingest_cli.py", line 185, in run_ingestion_and_check_upgrade
    ret = await ingestion_future
  File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/datahub/cli/ingest_cli.py", line 139, in run_pipeline_to_completion
    raise e
  File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/datahub/cli/ingest_cli.py", line 131, in run_pipeline_to_completion
    pipeline.run()
  File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/datahub/ingestion/run/pipeline.py", line 405, in run
    for wu in itertools.islice(
  File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/datahub/ingestion/api/source_helpers.py", line 171, in auto_workunit_reporter
    for wu in stream:
  File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/datahub/ingestion/api/source_helpers.py", line 253, in auto_browse_path_v2
    for urn, batch in _batch_workunits_by_urn(stream):
  File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/datahub/ingestion/api/source_helpers.py", line 391, in _batch_workunits_by_urn
    for wu in stream:
  File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/datahub/ingestion/api/source_helpers.py", line 184, in auto_materialize_referenced_tags
    for wu in stream:
  File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/datahub/ingestion/api/source_helpers.py", line 91, in auto_status_aspect
    for wu in stream:
  File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/datahub/ingestion/source/nifi.py", line 1048, in get_workunits_internal
    self.process_provenance_events()
  File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/datahub/ingestion/source/nifi.py", line 966, in process_provenance_events
    for event in events:
  File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/datahub/ingestion/source/nifi.py", line 674, in fetch_provenance_events
    provenance_response = self.submit_provenance_query(
  File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/datahub/ingestion/source/nifi.py", line 734, in submit_provenance_query
    older_version: bool = self.nifi_flow.version is not None and version.parse(
  File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/packaging/version.py", line 54, in parse
    return Version(version)
  File "/tmp/datahub/ingest/venv-nifi-2b9c1ab97dc6cd7f/lib/python3.10/site-packages/packaging/version.py", line 200, in __init__
    raise InvalidVersion(f"Invalid version: '{version}'")
packaging.version.InvalidVersion: Invalid version: '1.19.1-p2'