Skip to content

TPS Data crawler utilizing kotlin coroutine, the lightweight threading to create multi process to accelerate download process. Data result and analysises available at https://github.com/khrlimam/pemilu2024-suara-tps

Notifications You must be signed in to change notification settings

khrlimam/pemilu2024-crawler

Repository files navigation

This is the source code for TPS Crawler in kotlin. Built jar available here

Usage

Make sure it run on jdk 21

java -cp pemilu2024.main.jar proj.internal.MainKt --<options>

To crawl all tps

java -cp pemilu2024.main.jar proj.internal.MainKt --crawl

To crawl only wilayah

java -cp pemilu2024.main.jar proj.internal.MainKt --crawl --wilayah 11

where 11 is the wilayah code. This parameter is separated by "/" for nested wilayah e.g provinsi/kabupaten/kecamatan/lurah/tps max 5 level.

For example, to download all tps in kabupaten BIAK NUMFOR Papua, then:

java -cp pemilu2024.main.jar proj.internal.MainKt --crawl --wilayah 91/9106

This wilayah code can easily obtained from Info Publik Pemilu 2024 web page.

To download TPS from list of tps codes

java -cp pemilu2024.main.jar proj.internal.MainKt -tps ~/workspaces/pemilu2024/input/tps-anomali.txt -n anomali.csv --worker 1000

For other argument can use -h

About

TPS Data crawler utilizing kotlin coroutine, the lightweight threading to create multi process to accelerate download process. Data result and analysises available at https://github.com/khrlimam/pemilu2024-suara-tps

Topics

Resources

Stars

Watchers

Forks

Packages

No packages published

Languages