Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inconsistent CSV output #276

Open
mcauer opened this issue May 27, 2022 · 1 comment
Open

Inconsistent CSV output #276

mcauer opened this issue May 27, 2022 · 1 comment
Labels
bug Something isn't working

Comments

@mcauer
Copy link
Member

mcauer commented May 27, 2022

Bug Description

The way the CSV output is configured differs between endpoints.
Sometimes fields are quoted with double-quotes (/elements/count) and s.t. not (/elements/count/groupBy/boundary)
E.g.:

elements/count/

curl -X GET "https://api.ohsome.org/v1/elements/count?bboxes=8.67%2C49.39%2C8.71%2C49.42&filter=type%3Away%20and%20natural%3D*&format=csv&time=2014-01-01%2F2017-01-01%2FP1Y" -H  "accept: application/json"

# Copyright URL: https://ohsome.org/copyrights
# Copyright Text: © OpenStreetMap contributors
# API Version: 1.6.3
timestamp;value
"2014-01-01T00:00:00Z";"42.0"
"2015-01-01T00:00:00Z";"42.0"
"2016-01-01T00:00:00Z";"43.0"
"2017-01-01T00:00:00Z";"43.0"

elements/count/groupBy/boundary

curl -X GET "https://api.ohsome.org/v1/elements/count/groupBy/boundary?bboxes=8.67%2C49.39%2C8.71%2C49.42&filter=type%3Away%20and%20natural%3D*&format=csv&time=2014-01-01%2F2017-01-01%2FP1Y" -H  "accept: application/json"

# Copyright URL: https://ohsome.org/copyrights
# Copyright Text: © OpenStreetMap contributors
# API Version: 1.6.3
timestamp;boundary1
2014-01-01T00:00:00Z;42.0
2015-01-01T00:00:00Z;42.0
2016-01-01T00:00:00Z;43.0
2017-01-01T00:00:00Z;43.0

General Information

Please include the following general information about the issue and list any additional steps needed to reproduce the bug.

  • Version of the ohsome API : v1.6.3
  • Which API instance was requested: https://api.ohsome.org/v1`
  • Affected endpoint(s) /elements/count and /elements/count/groupbBy/boundary may be more
  • URL of your request (and request body if applicable): see above
  • Used HTTP method [GET or POST]: see above
  • Utilized tool/library for the request [e.g. cURL, Postman, ohsome-py, etc.]: see above

Expected Behaviour

Expected is that all ohsomeAPI CSV output follows the same encoding

Further Information

Error Messages, Logs, Screenshots

If applicable, add printed error messages, log files or screenshots to help explain your problem.

Additional Information

Add any other further information about the problem here.

Notes

Please consider to upload a file if request parameters, responses, error messages or logs are too long or poorly displayed in your bug report.

@mcauer mcauer added the bug Something isn't working label May 27, 2022
@joker234
Copy link
Member

The issue seems to be that the grouping results are created differently than the regular results, see

if (resultSet instanceof ElementsResult[]) {
ElementsResult[] rs = (ElementsResult[]) resultSet;
writer.writeNext(new String[] {"timestamp", "value"}, false);
for (ElementsResult elementsResult : rs) {
writer.writeNext(new String[] {elementsResult.getTimestamp(),
String.valueOf(elementsResult.getValue())});
}
vs.
} else if (resultSet instanceof GroupByResult[]) {
GroupByObject[] rs = (GroupByResult[]) resultSet;
if (resultSet.length == 0) {
writer.writeNext(new String[] {"timestamp"}, false);
} else {
var rows = createCsvResponseForElementsGroupBy(rs);
writer.writeNext(rows.getLeft().toArray(new String[rows.getLeft().size()]), false);
writer.writeAll(rows.getRight(), false);
}
}
and
private ImmutablePair<List<String>, List<String[]>> createCsvResponseForElementsGroupBy(
GroupByObject[] resultSet) {
List<String> columnNames = new LinkedList<>();
columnNames.add("timestamp");
List<String[]> rows = new LinkedList<>();
for (int i = 0; i < resultSet.length; i++) {
GroupByResult groupByResult = (GroupByResult) resultSet[i];
Object groupByObject = groupByResult.getGroupByObject();
if (groupByObject instanceof Object[]) {
Object[] groupByObjectArr = (Object[]) groupByObject;
columnNames.add(groupByObjectArr[0].toString() + "_" + groupByObjectArr[1].toString());
} else {
columnNames.add(groupByObject.toString());
}
for (int j = 0; j < groupByResult.getResult().length; j++) {
ElementsResult elemResult = (ElementsResult) groupByResult.getResult()[j];
if (i == 0) {
String[] row = new String[resultSet.length + 1];
row[0] = elemResult.getTimestamp();
row[1] = String.valueOf(elemResult.getValue());
rows.add(row);
} else {
rows.get(j)[i + 1] = String.valueOf(elemResult.getValue());
}
}
}
return new ImmutablePair<>(columnNames, rows);
}
.

@joker234 joker234 added this to To do in ohsome API general via automation May 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Development

No branches or pull requests

2 participants