Files
zkldi_Tachi/_scripts/make-datadumps.sh
T
zk 593adb2717 feat: Dev Container (#1149)
* temporary dvctr

* start of work

* fix: timeouts in windows

* fish?

* feat: better, working stuff

* refactor: rename database-seeds to seeds/

* better

* fdsaf

* feat: don't let our container get owned by root

* fix: better user handling

* feat: falling asleep at my desk i yield to you what i have done

* feat: autoadminify

* docs: denote trickshot

* feat: open in browser on launch

* better seeds workings...

* fix: dont kill shell

* docs: mention dev/debs

* fix: whoops

* fix: make seeds load within the century

* feat: rest of the containerisation

* feat: new alias

* style: fix linter errors

* fix: bad call

* fix: better justfile
2024-08-17 15:01:40 +01:00

28 lines
819 B
Bash
Executable File

#!/bin/bash
# a script for making tachi's anonymised datasets.
# designed to run on my machine
#
# you are expected to have pnpm, mongo etc. in path.
set -eox pipefail
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
cd "$SCRIPT_DIR";
cd ../server;
# stuff based on my local system, modify as wished
remote_port=12345
target=../../tachi-datasets/datasets
for kind in "kamai" "boku"; do
mongosh --eval "use anon-$kind" --eval "db.dropDatabase()"
mongodump --archive --port=$remote_port --db=$kind | mongorestore --archive --nsFrom="$kind.*" --nsTo="anon-$kind.*"
ts-node src/scripts/anonymise-db 127.0.0.1:27017/anon-$kind
TCHIS_CONF_LOCATION=$kind.dataset.conf.json5 pnpm set-indexes
mongodump --db=anon-$kind --archive="$target/anon-$kind-$(date +%Y-%m).dump" --gzip
done;