Skip to content

Update FirehoseJob.py#74

Merged
webcoderz merged 25 commits intomasterfrom
webcoderz-twint-patch-1
Aug 18, 2020
Merged

Update FirehoseJob.py#74
webcoderz merged 25 commits intomasterfrom
webcoderz-twint-patch-1

Conversation

@webcoderz
Copy link
Copy Markdown
Collaborator

updating firehose job for twint pool, and arrow statuses

@webcoderz webcoderz requested a review from lmeyerov June 30, 2020 17:57
@lmeyerov
Copy link
Copy Markdown
Contributor

Sounds like the missing patches from /twint folder?

Will approve, leave merge to your call + result of coding session

Twint will be pretty cool b/c going forward as we can do open source CI with end-to-end tests (neo4j, twint, ...) w/out needing twarc API keys :)

lmeyerov
lmeyerov previously approved these changes Jun 30, 2020
@webcoderz webcoderz requested a review from bmorphism July 1, 2020 04:03
lmeyerov
lmeyerov previously approved these changes Aug 16, 2020
&& rm -rf /var/lib/apt/lists/*

COPY ./app /app
RUN pip install prefect==0.10.1 simplejson twarc neo4j boto3==1.12.39 \
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

guessing we'll also need pandas/twint/bs4

@webcoderz webcoderz merged commit 04f9b2a into master Aug 18, 2020
@webcoderz webcoderz deleted the webcoderz-twint-patch-1 branch August 18, 2020 16:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants