Skip to content
Closed
Changes from 1 commit
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
ae2eb1d
adding is_tor flag in and setting in twintpool in checkhydrate proper…
webcoderz Sep 16, 2020
ff45ea0
adding is_tor flag in and setting in twintpool in checkhydrate proper…
webcoderz Sep 16, 2020
3c279f1
adding is_tor flag in and setting in twintpool in checkhydrate proper…
webcoderz Sep 16, 2020
10b241e
adding is_tor flag in and setting in twintpool in checkhydrate proper…
webcoderz Sep 16, 2020
14017fd
updating job
webcoderz Sep 16, 2020
3d7c969
updating job
webcoderz Sep 16, 2020
43b02d9
updating job
webcoderz Sep 16, 2020
9230edc
updating job
webcoderz Sep 16, 2020
88ed1ce
setting up datastream-Dockerfile for twitterscraper library
webcoderz Sep 24, 2020
06e322c
setting up datastream-Dockerfile for twitterscraper library
webcoderz Sep 24, 2020
9580d40
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 24, 2020
c36852b
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 24, 2020
a8ac909
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 24, 2020
5ecbedd
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 24, 2020
df9bd0b
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 24, 2020
2f84bcb
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 24, 2020
047430e
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 24, 2020
59d512b
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 24, 2020
a282d8d
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 24, 2020
540efe2
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 24, 2020
5ea76b8
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 24, 2020
58a66c7
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 24, 2020
7461321
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 24, 2020
332a4d9
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 24, 2020
431af04
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 24, 2020
07a7181
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 24, 2020
9d25744
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 24, 2020
c1e0eb9
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 24, 2020
a6983a4
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 24, 2020
10e600b
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 24, 2020
fc18003
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 24, 2020
0bd26b6
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 24, 2020
47297c5
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 24, 2020
84060b6
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 24, 2020
fad3bce
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 24, 2020
15463b4
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 25, 2020
dec74eb
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 25, 2020
631eb57
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 25, 2020
1aa8191
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 25, 2020
580a697
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 25, 2020
779cc99
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 25, 2020
b572c8e
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 25, 2020
0f0290f
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 25, 2020
a292cfe
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 25, 2020
b6a8ef3
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 25, 2020
2fed016
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 25, 2020
e862eef
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 25, 2020
b8e9757
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 25, 2020
39bc1e9
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
webcoderz Sep 25, 2020
badca9f
ublock extension
webcoderz Sep 26, 2020
b03e85b
ublock extension
webcoderz Sep 26, 2020
b269256
ublock extension
webcoderz Sep 26, 2020
2fa3400
fix
webcoderz Sep 26, 2020
e4ca790
fix
webcoderz Oct 2, 2020
dca3b1a
_get_user_timeline fix inline with twint updates
webcoderz Dec 26, 2020
c288c08
_get_user_timeline fix inline with twint updates
webcoderz Dec 26, 2020
31acfd9
_get_user_timeline fix inline with twint updates
webcoderz Dec 26, 2020
2aa9c2d
date time pipeline fix
webcoderz Dec 26, 2020
232c6b5
date time pipeline fix
webcoderz Dec 26, 2020
91b58b2
date time pipeline fix
webcoderz Dec 26, 2020
dee6da4
date time pipeline fix
webcoderz Dec 26, 2020
b39bfe3
added self.config.Hide_output=True
webcoderz Dec 26, 2020
798d9ee
user info inline with twint
webcoderz Dec 26, 2020
File filter

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
setting up datastream-Dockerfile for twitterscraper library w/ geckod…
…river
  • Loading branch information
webcoderz committed Sep 25, 2020
commit e862eef7a71b60da963d6ad7f39606378ee63b50
5 changes: 5 additions & 0 deletions infra/pipelines/docker/datastream-Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -33,10 +33,15 @@ RUN GK_VERSION=$(if [ ${GECKODRIVER_VERSION:-latest} = "latest" ]; then echo "0.
&& tar -C /opt -zxf /tmp/geckodriver.tar.gz \
&& rm /tmp/geckodriver.tar.gz \
&& mv /opt/geckodriver /opt/geckodriver-$GK_VERSION \
&& cp /opt/geckodriver-$GK_VERSION /bin \
&& chmod 755 /opt/geckodriver-$GK_VERSION \
&& ln -fs /opt/geckodriver-$GK_VERSION /usr/bin/geckodriver \
&& ln -fs /opt/geckodriver-$GK_VERSION /usr/bin/wires


ENV PATH="${PATH}:/opt/geckodriver-0.27.0"


RUN pip install prefect==0.10.1 simplejson twarc neo4j boto3==1.12.39 \
pandas pyarrow urlextract git+https://github.com/lmeyerov/twint.git@patch-1#egg=twint \
git+https://github.com/lapp0/twitterscraper.git@selenium
Expand Down