Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

data extraction: standardise column names for log exports #649

Open
aislinnpearson opened this issue Apr 19, 2024 · 1 comment
Open

data extraction: standardise column names for log exports #649

aislinnpearson opened this issue Apr 19, 2024 · 1 comment
Labels
bug Something isn't working Data extraction Issues relating to extracting data from FarmOS Data Model Issues which require a change to the data model used by Rothamsted FarmOS Core Issues relating to FarmOS Core
Milestone

Comments

@aislinnpearson
Copy link
Collaborator

When I downloading the logs from the classical experiments so Tony could compile the yield books, I noticed that where we (Rothamsted) have extended the underlying data model for the logs, the naming of the columns is inconsistent.

The data fields I noticed it for were mainly PPE, COSHH and seed dressings where all three of these things would be amalgamated into the same two columns. There were also examples of additional rows for lot number, method and purchase source in some exports but not others, which were always empty columns and which also I wasn’t sure about. I’ve attached two examples here to demonstrate what I mean.

@aislinnpearson aislinnpearson added Data Model Issues which require a change to the data model used by Rothamsted Data extraction Issues relating to extracting data from FarmOS labels Apr 19, 2024
@aislinnpearson aislinnpearson added this to the Queued Issues milestone Apr 19, 2024
@aislinnpearson aislinnpearson added this to Issues for Triage in Feature requests via automation Apr 19, 2024
@paul121 paul121 added bug Something isn't working FarmOS Core Issues relating to FarmOS Core labels Apr 22, 2024
@paul121
Copy link
Collaborator

paul121 commented Apr 22, 2024

Drat. This is a known issue I had forgotten about. farmOS/farmOS#805

Headers are generated for the first entity in the export list. When exporting many assets/logs of different bundles this means some entities may have different fields than others. It is indeed a bug that the resulting export may have incorrect column names for some of the row data.

For example, Harvest logs have a "Storage location" field, while other Input logs have "COSH" and "PPE" fields. When exporting a CSV of both Harvest and Input logs, these data fields/headers are not consistent with row data in the CSV exports.

@paul121 paul121 modified the milestones: Queued Issues, 2.21.0 Apr 22, 2024
@paul121 paul121 modified the milestones: 2.21.0, 2.22.0 May 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Data extraction Issues relating to extracting data from FarmOS Data Model Issues which require a change to the data model used by Rothamsted FarmOS Core Issues relating to FarmOS Core
Projects
Feature requests
Issues for Triage
Development

No branches or pull requests

2 participants