Skip to content

Chrome extension to "Create WARC files from any webpage"

License

Notifications You must be signed in to change notification settings

machawk1/warcreate

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

WARCreate logo
WARCreate

"Create WARC files from any webpage"


WARCreate is a Google Chrome extension with an aim to be able to "Create WARC files from any webpage".

With WARCs normally being limited to be generated by Internet Archive's Heritrix Archival Crawler, providing another means of generating these files from webpages opens the door to

  • Preserving content not accessible to crawlers (e.g., deep web contents)
  • Circumventing the complication and overhead needed to setup a Heritrix instance by an end-user
  • Allowing a webpage to be interacted with (e.g., Facebook comments unrolled) prior to preservation, ensuring content that might not be initially present in a page is available to be captured.

...among many other use cases.

WARCreate is currently in active development though has gone through various release and retraction periods due to changes in the Google Chrome extension API and rules controlling extension distribution.

The original idea and prototype was published in the Joint Conference on Digital Libraries 2012 (JCDL '12) Proceedings.

Install

The latest stable binary can be downloaded from the Chrome Web Store.

Citing Project

A publication related to this project appeared in the proceedings of JCDL 2012 (Read the PDF). Please cite it as below:

Mat Kelly and Michele C. Weigle. WARCreate - Create Wayback-Consumable WARC Files from Any Webpage. In Proceedings of the ACM/IEEE Joint Conference on Digital Libraries (JCDL), pages 437–438, Washington, DC, June 2012.

@INPROCEEDINGS{warcreate-jcdl2012,
  AUTHOR    = {Mat Kelly and
               Michele C. Weigle},
  TITLE     = {{WARCreate} - Create Wayback-Consumable WARC Files from Any Webpage},
  BOOKTITLE = {Proceedings of the ACM/IEEE Joint Conference on Digital Libraries (JCDL)},
  PAGES     = {437--438},
  MONTH     = {June},
  YEAR      = {2012},
  ADDRESS   = {Washington, DC},
  DOI       = {10.1145/2232817.2232930}
}

Contact

WARCreate is a project of the Web Science and Digital Libraries (WS-DL) research group at Old Dominion University (ODU), created by Mat Kelly.

For support e-mail warcreate@matkelly.com or tweet to us at @machawk1 and/or @WebSciDL.

License

MIT

About

Chrome extension to "Create WARC files from any webpage"

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 4

  •  
  •  
  •  
  •