Using Python to access tweets from the command line

Here at the Harvard of the Plains, I teach a class in digital product development I like to call Programming as an Act of Journalism. Lots of people ask me about it and I’m always a bit cagey about it because, to be frank, I’m still kind of making it up as I go. My course goals could actually fill an entire degree, so I spend a lot of time pushing and pulling against my wants and needs for the class. But the basic outline is I take College of Journalism students who know nothing about code and product development and we build a prototype of a product they invent in 16 weeks. And the first eight weeks of hands-on classes are spent doing intro to programming type work. 

I’ve started developing lessons I call Small Wins for the class so they can see stuff working on the screen. Here’s the first: a lesson in using Python to access the Twitter API. It’s the follow up class to basic Python (variables, strings, integers, functions and lists). The goal is to talk about using libraries, introduce programmatic thinking and data structure and, well, see some stuff happen in front of their eyes.

In the class, I have students install a virtual machine and put Ubuntu Linux on that. Why? Lots of reasons, but the main ones are that I want to expose students to a new environment while at the same time keeping it compartmentalized so they don’t fear “messing up their computer” by working in the terminal. So this walkthrough assumes you’re using Ubuntu.

1. Install pip, a package management library for Python.

$ sudo apt-get install python-pip python-dev build-essential 

2. Install python-twitter, a Python wrapper around the Twitter API that makes accessing the API breathtakingly simple (really!).

$ sudo pip install python-twitter

Start Python

$ python

Create an instance of the Twitter API class, which creates an object with several methods that we’re going to be using. Think of it like buying a Swiss Army knife. You get the knife and a whole bunch of tools that go with it. You just have to unfold them to use them.

>>> import twitter

>>> api = twitter.Api()

5. Pick a Twitter user to gather their tweets. For this example, we’ll use Stephen Colbert (@StephenAtHome). Then we’ll use one of our tools in the Twitter API class — GetUserTimeline, which does what you think it does.

>>> statuses = api.GetUserTimeline('StephenAtHome')

6. Congrats. You now have Stephen Colbert’s latest tweets. Stand in appreciation of what all the python-twitter library has done for you. It created an http request, sent that request to the properly formatted API URL, ingested the response, converted the structured JSON file it returned into a Python dictionary and returned it to you as an object for you to use. What’s that? You’re not excited? You want to see them? Okay, fine.

>>> for tweet in statuses:

...     print tweet

Yikes. What is that? It’s all the data that comes with each tweet. Take a look at it all. Background colors, times, dates, all kinds of stuff. Well, that won’t do. Maybe we just want to see the text of the tweet. If you look at the output, you’ll see a pattern in the output. You’ll see things like “protected”: false and “screen_name”: “StephenAtHome”. Those are called key/value pairs. The key is the name of the attribute — like an ID or the text of the tweet — and the value is what it sounds like it is.

7. In our case, we want to see the text of the tweet. To do that, you have to address the key. In Python, we do that with dot notation. What does that mean? It means you can pick the key in an object and display it by adding a dot after your object and then the key name. So like this: nameofobject DOT name of key

>>> for tweet in statuses:

...     print tweet.text


— Looking at the keys, what would you have to do to see if each tweet was favorited?

8. Okay, one users tweets are nice, but what if I want to see tweets about a subject, not a person? For that, we’ll use a different function in the python-twitter library, GetSearch:

>>> tweets = api.GetSearch("politifact", per_page=100, page=1)


— Using what you’ve learned, how would you print out the text of the tweets gathered in step 8?

Thoughts, suggestions, criticisms? The comment box awaits you. Want to go further with this? Here’s the python-twitter code and documentation