Crontab with Selenium and Chrome Driver

This is how to setup a crontab with Selenium and Chrome Driver.

Setup

  • Linux Ubuntu 16.04 with GNOME
  • Vim
  • Python 2.7.12
  • Bash 4.3.46

If you are here I assume you have some knowledge of cron. Otherwise read this cronjob tutorial.

Let’s pretend that we have a Python script in this directory:

~/Documents/scripts/awesome.py

Just for reference. You should use this for troubleshooting purposes:

Review the system log file here: /var/log/syslog

Send the stderr and stdout to a file like this. Right after the last crontab command:

> awesome.log 2>&1

Read more about the redirection syntax here.

Review your Python script

To be semantically correct, your header should have this:

#!/usr/bin/env python

Edit your crontab

Edit your crontab with crontab -e

In this example you run a job every day at 7:45am.

45 7 * * * export DISPLAY=:0 && cd /home/your-username/Documents/scripts/ && /usr/bin/python awesome.py > awesome.log 2>&1

To learn what 45 7 * * * means. Go to this tutorial about automating tasks with crontab.

Since cron runs with a limited number of environment variables. You need to set the DISPLAY variable.

This explains more about the DISPLAY variable.

DISPLAY consists of a keyboard, a mouse and a screen. The display is managed by the X server.

If I want to open Sublime in Linux. X server is the framework that helps me display the program into a window. If I move the mouse then it helps me go to an exact pixel in the screen.

Lookup more details on Wikipedia as X Window System.

The X server receives input from the keyboard and mouse and displays to a screen. A web browser and a terminal run on the user’s workstation.

If you do this on your terminal:

echo $DISPLAY

Most likely you will get:

:0

Which means that DISPLAY variable is set to :0.

If you do this on your terminal:

env | grep 'DISPLAY'

You will get:

DISPLAY:=0

Other examples of DISPLAY could be:

  • DISPLAY=localhost:4
  • DISPLAY=google.com:0

The syntax for DISPLAY is hostname:D.S which means screen S on display D of host hostname.

hostname:D.S

  • hostname is the computer name where the X server is running. If there is no hostname then it means the localhost
  • D is a sequence number. Usually 0. It can be different if the computer has many displays connected.
  • S is the screen number. 0 is the default.

When you echo $DISPLAY. The hostname is omitted and it shows only the default DISPLAY number: :0.

Crontab and DISPLAY

Selenium opens the browser if you are using the Chromedriver.

Cron doesn’t have the DISPLAY environment variable set.

You need to set the DISPLAY variable inside the crontab such as:

export DISPLAY=:0

Going back the example where you run a job every day at 7:45am.

45 7 * * * export DISPLAY=:0 && cd /home/your-username/Documents/scripts/ && /usr/bin/python awesome.py > awesome.log 2>&1

You are setting the DISPLAY variable before executing any command.

You are using the full path to where the files are.

And you are sending stdout and stderr to a log file for troubleshooting.

Leave a Reply