{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":313997425,"defaultBranch":"main","name":"wbtools","ownerLogin":"WormBase","currentUserCanPush":false,"isFork":false,"isEmpty":false,"createdAt":"2020-11-18T16:46:57.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/610960?v=4","public":true,"private":false,"isOrgOwned":true},"refInfo":{"name":"","listCacheKey":"v0:1711501592.0","currentOid":""},"activityList":{"items":[{"before":"30ef5b3cff8fcc9a3b154391e1d1156d812dd8ea","after":"bb7e65162763991fa7a696930cbef419ce646ca8","ref":"refs/heads/main","pushedAt":"2024-04-02T22:30:53.000Z","pushType":"push","commitsCount":3,"pusher":{"login":"valearna","name":"Valerio Arnaboldi","path":"/valearna","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/26348851?s=80&v=4"},"commit":{"message":"Added missing return","shortMessageHtmlLink":"Added missing return"}},{"before":"6ce98f8e7be72d3942d1cecb1ca4d1be60e6292c","after":"30ef5b3cff8fcc9a3b154391e1d1156d812dd8ea","ref":"refs/heads/main","pushedAt":"2024-03-27T19:38:14.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"valearna","name":"Valerio Arnaboldi","path":"/valearna","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/26348851?s=80&v=4"},"commit":{"message":"Including figure legends","shortMessageHtmlLink":"Including figure legends"}},{"before":"ae1e12afee6888f86e90ed040c40fbbc9ce6b94e","after":"6ce98f8e7be72d3942d1cecb1ca4d1be60e6292c","ref":"refs/heads/main","pushedAt":"2024-03-27T18:43:23.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"valearna","name":"Valerio Arnaboldi","path":"/valearna","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/26348851?s=80&v=4"},"commit":{"message":"Refined section filter conditions","shortMessageHtmlLink":"Refined section filter conditions"}},{"before":"693f147b8abaeb2cf52e1049ddb8fbb63215e613","after":"ae1e12afee6888f86e90ed040c40fbbc9ce6b94e","ref":"refs/heads/main","pushedAt":"2024-03-27T18:25:30.000Z","pushType":"push","commitsCount":2,"pusher":{"login":"valearna","name":"Valerio Arnaboldi","path":"/valearna","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/26348851?s=80&v=4"},"commit":{"message":"Added more conditions to exclude references and additional sections","shortMessageHtmlLink":"Added more conditions to exclude references and additional sections"}},{"before":"d80af48a6f2c266fa3f7107ca4aa35df9290a42c","after":"ae1e12afee6888f86e90ed040c40fbbc9ce6b94e","ref":"refs/heads/develop","pushedAt":"2024-03-27T18:24:57.000Z","pushType":"push","commitsCount":3,"pusher":{"login":"valearna","name":"Valerio Arnaboldi","path":"/valearna","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/26348851?s=80&v=4"},"commit":{"message":"Added more conditions to exclude references and additional sections","shortMessageHtmlLink":"Added more conditions to exclude references and additional sections"}},{"before":"475a8303fbcf9e4e1e62e6256d183a23e3853d70","after":"693f147b8abaeb2cf52e1049ddb8fbb63215e613","ref":"refs/heads/main","pushedAt":"2024-03-27T01:15:22.000Z","pushType":"push","commitsCount":2,"pusher":{"login":"valearna","name":"Valerio Arnaboldi","path":"/valearna","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/26348851?s=80&v=4"},"commit":{"message":"New version number","shortMessageHtmlLink":"New version number"}},{"before":null,"after":"d80af48a6f2c266fa3f7107ca4aa35df9290a42c","ref":"refs/heads/develop","pushedAt":"2024-03-27T01:06:32.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"valearna","name":"Valerio Arnaboldi","path":"/valearna","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/26348851?s=80&v=4"},"commit":{"message":"Removing xml tags from sentences","shortMessageHtmlLink":"Removing xml tags from sentences"}},{"before":"b6e43a65b9e10c3f2f120e2f038d7bd2e24c5785","after":"475a8303fbcf9e4e1e62e6256d183a23e3853d70","ref":"refs/heads/main","pushedAt":"2024-03-26T21:22:04.000Z","pushType":"push","commitsCount":12,"pusher":{"login":"valearna","name":"Valerio Arnaboldi","path":"/valearna","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/26348851?s=80&v=4"},"commit":{"message":"Merge branch 'main' into grobid","shortMessageHtmlLink":"Merge branch 'main' into grobid"}},{"before":"d782fbb66a92d192f06ef7bdf792c748750693fb","after":"475a8303fbcf9e4e1e62e6256d183a23e3853d70","ref":"refs/heads/grobid","pushedAt":"2024-03-26T21:20:48.000Z","pushType":"push","commitsCount":2,"pusher":{"login":"valearna","name":"Valerio Arnaboldi","path":"/valearna","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/26348851?s=80&v=4"},"commit":{"message":"Merge branch 'main' into grobid","shortMessageHtmlLink":"Merge branch 'main' into grobid"}},{"before":"7f8fc410d8226f1c7f6d7f6dba0ec8db4d8d1c2c","after":"b6e43a65b9e10c3f2f120e2f038d7bd2e24c5785","ref":"refs/heads/main","pushedAt":"2024-03-14T16:27:44.000Z","pushType":"push","commitsCount":2,"pusher":{"login":"valearna","name":"Valerio Arnaboldi","path":"/valearna","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/26348851?s=80&v=4"},"commit":{"message":"New version number","shortMessageHtmlLink":"New version number"}},{"before":"8bcbeecbd44f0fbb05d4ab1b7a3a1788f1e2b64b","after":"d782fbb66a92d192f06ef7bdf792c748750693fb","ref":"refs/heads/grobid","pushedAt":"2024-03-14T16:23:02.000Z","pushType":"push","commitsCount":2,"pusher":{"login":"valearna","name":"Valerio Arnaboldi","path":"/valearna","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/26348851?s=80&v=4"},"commit":{"message":"Merge branch 'main' into grobid\n\n# Conflicts:\n#\twbtools/literature/corpus.py\n#\twbtools/literature/paper.py","shortMessageHtmlLink":"Merge branch 'main' into grobid"}},{"before":null,"after":"c1ecbdbb980ec0f6adfd521c804f7cc475d260b9","ref":"refs/heads/biblio_from_abc","pushedAt":"2024-03-14T16:18:34.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"valearna","name":"Valerio Arnaboldi","path":"/valearna","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/26348851?s=80&v=4"},"commit":{"message":"Removed obsolete text polishing","shortMessageHtmlLink":"Removed obsolete text polishing"}},{"before":"8be80ecd3b31c011e6de28c9c59f00b29aa463a9","after":"8bcbeecbd44f0fbb05d4ab1b7a3a1788f1e2b64b","ref":"refs/heads/grobid","pushedAt":"2024-03-01T17:32:08.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"valearna","name":"Valerio Arnaboldi","path":"/valearna","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/26348851?s=80&v=4"},"commit":{"message":"Removed obsolete text polishing","shortMessageHtmlLink":"Removed obsolete text polishing"}},{"before":"3275f8d8f454d80c377d73d09a0a1bcf29bed20c","after":"8be80ecd3b31c011e6de28c9c59f00b29aa463a9","ref":"refs/heads/grobid","pushedAt":"2024-03-01T17:28:16.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"valearna","name":"Valerio Arnaboldi","path":"/valearna","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/26348851?s=80&v=4"},"commit":{"message":"Use GROBID sentence extraction","shortMessageHtmlLink":"Use GROBID sentence extraction"}},{"before":"5ab4d040ae792ced7fc11afb1e19efdde269032f","after":"3275f8d8f454d80c377d73d09a0a1bcf29bed20c","ref":"refs/heads/grobid","pushedAt":"2024-02-21T17:56:38.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"valearna","name":"Valerio Arnaboldi","path":"/valearna","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/26348851?s=80&v=4"},"commit":{"message":"Using env var for GROBID API URL","shortMessageHtmlLink":"Using env var for GROBID API URL"}},{"before":"c719b50fd1d777f026671fee7f8975daa05f4d38","after":"5ab4d040ae792ced7fc11afb1e19efdde269032f","ref":"refs/heads/grobid","pushedAt":"2024-02-21T17:38:51.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"valearna","name":"Valerio Arnaboldi","path":"/valearna","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/26348851?s=80&v=4"},"commit":{"message":"Fixed parentheses","shortMessageHtmlLink":"Fixed parentheses"}},{"before":"3385f5a886f1ced55a3a830ef1a8eb95d9c01e24","after":"c719b50fd1d777f026671fee7f8975daa05f4d38","ref":"refs/heads/grobid","pushedAt":"2024-02-21T17:36:53.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"valearna","name":"Valerio Arnaboldi","path":"/valearna","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/26348851?s=80&v=4"},"commit":{"message":"Fixed bug\n\nref_file_mod was accessed instead of ref_file","shortMessageHtmlLink":"Fixed bug"}},{"before":"0f9fbd73476ac3a45340ee9112d5a0504d01c47b","after":"3385f5a886f1ced55a3a830ef1a8eb95d9c01e24","ref":"refs/heads/grobid","pushedAt":"2024-02-21T17:33:41.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"valearna","name":"Valerio Arnaboldi","path":"/valearna","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/26348851?s=80&v=4"},"commit":{"message":"Added option to load main file only","shortMessageHtmlLink":"Added option to load main file only"}},{"before":"95c90d0e843ecdd2594b1156121537336918b729","after":"0f9fbd73476ac3a45340ee9112d5a0504d01c47b","ref":"refs/heads/grobid","pushedAt":"2024-02-21T17:24:41.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"valearna","name":"Valerio Arnaboldi","path":"/valearna","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/26348851?s=80&v=4"},"commit":{"message":"Removed extra .","shortMessageHtmlLink":"Removed extra ."}},{"before":"13f89e8d27b2ddcd0f3ffdae3999e23f3b032f3d","after":"95c90d0e843ecdd2594b1156121537336918b729","ref":"refs/heads/grobid","pushedAt":"2024-02-20T20:33:40.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"valearna","name":"Valerio Arnaboldi","path":"/valearna","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/26348851?s=80&v=4"},"commit":{"message":"Updated version number","shortMessageHtmlLink":"Updated version number"}},{"before":"0ea3ed3c1148e0593c6c43a4448ee5cbf668c67d","after":"13f89e8d27b2ddcd0f3ffdae3999e23f3b032f3d","ref":"refs/heads/grobid","pushedAt":"2024-02-20T20:28:46.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"valearna","name":"Valerio Arnaboldi","path":"/valearna","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/26348851?s=80&v=4"},"commit":{"message":"Fixed indentation and removed assert","shortMessageHtmlLink":"Fixed indentation and removed assert"}},{"before":null,"after":"0ea3ed3c1148e0593c6c43a4448ee5cbf668c67d","ref":"refs/heads/grobid","pushedAt":"2024-02-20T20:21:14.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"valearna","name":"Valerio Arnaboldi","path":"/valearna","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/26348851?s=80&v=4"},"commit":{"message":"First test with grobid for pdf2txt conversion","shortMessageHtmlLink":"First test with grobid for pdf2txt conversion"}},{"before":"c27e3c5b55e20291e2374608ec7a60cc230176cb","after":"7f8fc410d8226f1c7f6d7f6dba0ec8db4d8d1c2c","ref":"refs/heads/main","pushedAt":"2023-10-16T00:52:56.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"valearna","name":"Valerio Arnaboldi","path":"/valearna","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/26348851?s=80&v=4"},"commit":{"message":"Bump version and improve data handling in literature module\n\nUpdate version in setup.py from 2.0.2 to 2.0.3 to mark new release. In literature/corpus.py, add check for None type 'agr_curie' to prevent errors during data processing. Adjust SQL query in db/paper.py to return None when no matching record is found, improving data robustness.","shortMessageHtmlLink":"Bump version and improve data handling in literature module"}},{"before":"7bccbea43a2dbab0c361b10ca6db511554c9f6fe","after":"c27e3c5b55e20291e2374608ec7a60cc230176cb","ref":"refs/heads/main","pushedAt":"2023-10-14T22:16:18.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"valearna","name":"Valerio Arnaboldi","path":"/valearna","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/26348851?s=80&v=4"},"commit":{"message":"Update version and SQL query in setup.py and generic.py\n\nIncreased version number from 2.0.1 to 2.0.2 in setup.py to reflect recent changes. In generic.py, modified the SQL query to select from 'pap_title' instead of 'pap_electronic_path' for better dataset retrieval based on timestamp.","shortMessageHtmlLink":"Update version and SQL query in setup.py and generic.py"}},{"before":"b36866bc23c95550948680657067e98f687554a9","after":"7bccbea43a2dbab0c361b10ca6db511554c9f6fe","ref":"refs/heads/main","pushedAt":"2023-10-12T20:20:11.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"valearna","name":"Valerio Arnaboldi","path":"/valearna","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/26348851?s=80&v=4"},"commit":{"message":"Add 'requests' and 'python-dateutil' to setup.py\n\nTwo new dependencies ('requests' and 'python-dateutil') were added to the setup.py file. These were necessary to properly handle HTTP requests and provide more flexibility when dealing with dates and times in the pdfminer.six and pytz modules respectively.","shortMessageHtmlLink":"Add 'requests' and 'python-dateutil' to setup.py"}},{"before":"ac51fb521e0a04337f3eee44f88960b03fb0a7b4","after":"b36866bc23c95550948680657067e98f687554a9","ref":"refs/heads/main","pushedAt":"2023-10-12T20:19:45.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"valearna","name":"Valerio Arnaboldi","path":"/valearna","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/26348851?s=80&v=4"},"commit":{"message":"Add 'requests' and 'python-dateutil' to setup.py\n\nTwo new dependencies ('requests' and 'python-dateutil') were added to the setup.py file. These were necessary to properly handle HTTP requests and provide more flexibility when dealing with dates and times in the pdfminer.six and pytz modules respectively.","shortMessageHtmlLink":"Add 'requests' and 'python-dateutil' to setup.py"}},{"before":"09db3ce1cddcc54fe9d38c291939146d081dadcf","after":"ac51fb521e0a04337f3eee44f88960b03fb0a7b4","ref":"refs/heads/main","pushedAt":"2023-10-12T19:52:16.000Z","pushType":"push","commitsCount":4,"pusher":{"login":"valearna","name":"Valerio Arnaboldi","path":"/valearna","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/26348851?s=80&v=4"},"commit":{"message":"Update version number in setup.py\n\nThe version number was increased from 1.3.6 to 2.0.0 to reflect significant changes and improvements incorporated into the wbtools package. This includes switching to Alliance ABC as source of truth for PDFs.","shortMessageHtmlLink":"Update version number in setup.py"}},{"before":"a0e13af039c9396fedbe3f97eed31bed7c882a08","after":"b86de2ffd348d7eda9f08051dd3eb8ae8cffd6fc","ref":"refs/heads/abc_pdf_source","pushedAt":"2023-10-12T19:47:27.000Z","pushType":"push","commitsCount":2,"pusher":{"login":"valearna","name":"Valerio Arnaboldi","path":"/valearna","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/26348851?s=80&v=4"},"commit":{"message":"Remove unused text file in tests/data/text_files\n\nThe files were deleted from the tests/data/text_files directory as they are no longer needed for any testing purposes. This step improves the cleanliness and organization of test files within the project.\n\nAll tests have been fixed with the new ABC PDF source of truth","shortMessageHtmlLink":"Remove unused text file in tests/data/text_files"}},{"before":null,"after":"a0e13af039c9396fedbe3f97eed31bed7c882a08","ref":"refs/heads/abc_pdf_source","pushedAt":"2023-10-12T00:06:01.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"valearna","name":"Valerio Arnaboldi","path":"/valearna","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/26348851?s=80&v=4"},"commit":{"message":"Add support for Alliance API to fetch, read and process PDF files\n\nReorganized the methods to read PDF data from Alliance API in wbtools/literature/paper.py. The PDF content is accessed from the API, read, and added to the respective paper documents based on the file type. This method removes the dependency on a remote file server. The Alliance API access uses reference curies, now in the caltech db and read from db/generic.py enabling paper id association with curie id to enable the API calls.\n\nChanges were made to tests/literature/test_paper.py, tests/literature/test_corpus.py and tests/config_reader.py to accommodate the new methods and verify their proper functioning.\n\nA new file wbtools/utils/okta_utils.py was added to generate headers for the Alliance API calls, get an authentication token, and update the token when needed.\n\nIn wbtools/literature/corpus.py, the method to load text from PDF files from the Alliance API is incorporated.\n\nThe requirements.txt was updated to add the necessary packages for the Alliance API calls and dotenv package to handle environment variables.\n\nThe wbtools/literature/paper.py file was tidied up to remove the unnecessary and redundant lines of code to make it more compact and efficient.\n\nAll these modifications were done to ensure the PDFs are accessed directly from the Alliance API, enhancing the efficiency and real-time data access for the curation process.","shortMessageHtmlLink":"Add support for Alliance API to fetch, read and process PDF files"}},{"before":"b90875792bd305eb1fe88c7c23939ae4f9c7f230","after":"09db3ce1cddcc54fe9d38c291939146d081dadcf","ref":"refs/heads/main","pushedAt":"2023-10-05T19:27:40.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"valearna","name":"Valerio Arnaboldi","path":"/valearna","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/26348851?s=80&v=4"},"commit":{"message":"Update curation URL and increment version in setup.py\n\nRevised the request URLs in scraping.py to resolve to 'https://caltech-curation.textpressolab.com/priv/', enhancing security by avoiding direct access to sensitive directories. Also, incremented the version in setup.py to 1.3.6 signifying the new changes, ensuring proper version control.","shortMessageHtmlLink":"Update curation URL and increment version in setup.py"}}],"hasNextPage":true,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAAEJgj17QA","startCursor":null,"endCursor":null}},"title":"Activity · WormBase/wbtools"}