Sample Articles:

I Am Ebooks (And So Can You!)

Introduction

It feels like yesterday to me but about a year ago these strange accounts started showing up on my twitter feed. They looked and acted like some of the people I followed but their icons were flipped upside down. If you talked to them, within 30 seconds they'd talk back and they updated their status nearly every hour day or night. I soon realized that they were not people but rather elaborate twitter robots designed to mimic the speech of an existing user. It didn't take long before my curiosity and fascination got the better of me and I started learning ruby so I could program one of my own. One led to another, to another, to another, eventually I started getting all kinds of ideas about what I could make a bot do. To celebrate a year of this dark hole I fell into, I want to share some of my knowledge with novice users and people, like me, who have never touched ruby before trying to make a twitter bot.

Basic Concepts

If you know what ruby is and what a Markov chain is, you could totally skip this, but for new users I want to briefly cover these two topics in an unsubstantive way that will piss off experienced programmers.

If you're new to ruby, the best place to start is actually not here but with a guide called: Why's (poignant) Guide to Ruby. You don't need to read all of it, nor do I expect you to, however the first couple chapters are very helpful in discussing the very fluid, English nature, of the Ruby programming language. This is not a necessity, but if you're attempting to create advanced commands, especially call and response style actions, it helps to know the structure of the language.

Apart from the programming language, what makes a bot so special? A lot of the appeal and uniqueness to these bots comes from Markov Chains. Most _ebooks bots user Markov chains to generate text, that is to say that they use math to figure out language. A Markov chain is a mathematical system that as it transitions from one state to another state undergoes a random memory-less process that is dependent on the first state. Wikipedia bastardization aside, what that means is that given one state there are statistical, predictable, outcomes for what should follow. As ebooks bots do this with English sentences, let's do the same. We can start with any word, but let's start with "I". What should be the next word? How often do we follow "I" with something? Let's say there is a 70% chance we will follow "I" with "am" and a 30% chance that we follow "I" with "was". We have a simple binary choice now to start the beginning of a sentence. "I am" "I was". What follows either of those? Let's say "I am" is followed by "thirsty" 30% of the time, "hungry", 40% of the time, and "going" 30% of the time. "I was" is followed by "going" 50% of the time, and "inebriated" 50% of the time.

We can start to statistically model sentences around this. I was going to die. I am going to the store. I am hungry for sushi. I was completely intoxicated on wine last night (true fact!). Now, let's compile these, and hundreds of other sentences, into a single file. That file is our **corpus**, the language element of our bot, that will be consumed. Consuming the corpus determines probability, how often one word follows another, and what words we start sentences with, which gets stored as a text model. Your first corpus will only be 3000 tweets, but you can update this corpus constantly. I believe my personal bot may have a corpus of 25,000 tweets, so she almost has my language and tone absolutely nailed. Your bot may have fewer, or it may have more, depending on how you build your corpus and what your source is. If this is your first time making something like this, start small with an actual human being. As you begin to understand the workings of your bot, feel free to start thinking outside of the box and incorporating other sources. I have a bot that prints nothing but Julius Caesar's dialogue from the Shakespeare play, a friend of mine has a bot whose corpus is text dumps from anime subtitles. If you find a large amount of text somewhere, chances are it would make a good corpus.

This guide would make a good corpus.

Setting up the Account

Let's take a few minutes to set up the account. There's a few things you're going to need, the first is a shiny new email from some random ass provider for your bot. Here's mine:

Fill in the silly capchas and get that account running. You will be using it solely to verify the twitter account. After that the email is kinda pointless unless you love receiving spam.

Next, you need to set up the twitter account itself. You may notice in the screenshots that I'm using my browser in private browsing mode. If you have a main twitter account that's two-step authenticated or you want to access it while you work on your bot, use private browsing.

Once you set up the account, go to the main twitter screen.

Follow Rick Mercer, no wait. Don't do that.

Here you should quickly set up your picture, header, and description. You can do that later if you want, but honestly you won't be poking around too much with the account and this is the first and only time you should be on this screen.

Confirm your email and move on.

This is how my example bot's page looks:

Installing Ruby

If you're working on OS X, BSD, or Linux, open up the Terminal and type:

If you see a ruby version number appear, excellent. If not, there are multiple ways of installing ruby on these systems. Head to https://rvm.io/rvm/install and follow the instructions. Once rvm is set up, type this command:

or

I'm using the newer version for this example but as you'll see later, this requires modifying the Gemfile to work. Either version should work.

If you're a windows user, sorry I can't be more helpful here. There is a .exe binary out there available here: http://rubyinstaller.org/

It should be usable via the command line and will work similarly to the versions above but I have NO experience with it. Grab 2.1.3 if it's available.

Once you've got ruby working. Type this command to get the twitter_ebooks gem.

After about 30 seconds and a pile of output you'll have all the gems that are required to run the bot. 

Navigate to where you want the folder to go and type this command.

Making the App

Let's do a test here:

You will need to run this command eventually so give it a shot here.

Replace the <username> with whatever username you are basing the ebooks bot on.

Trying to archive my main account resulted in output asking for my Oauth tokens. To get these we will need to create a twitter application.

Go to https://apps.twitter.com and create your ebooks app. It's best practice to do this on your main account or at least any account with a phone number attached. You WILL need to be phone verified to create a "read/write" app, mostly because of people like me. Fill in the details accordingly, the app will need to have a unique name and a description a hell of a lot longer than my example. <account name> Ebooks App is good if you're only making one bot, if you're going to make a fleet of bots name it something like "Pira's magical bot app."

We've agreed to whatever it is in that long text that we will never read and now we have an app.

BAM! Now we just need to generate the Oauth tokens for this app and change it to read/write. Here we see our Customer Key and Customer Secret Key. You will need those for later so save them in a simple textfile somewhere. They're going in your bots.rb

Scroll down towards the bottom and click on Create my access token.

This will give you 2 more oauth tokens for your main account, these two will be used for the ebooks gem and could be used to control your account if you wanted to automate parts of it with twitter_ebooks.

Put those tokens into the same textfile as the previous two. You'll need to copy and paste all 4 very shorty. Move to the top of the page and click on the permissions tab.

Give your app Read, Write, and Access direct messages permissions and return back to your terminal or command line. Enter your four tokens and rerun your command. It should work wonderfully.

I pulled in two archives which is for something silly I'm doing. You only need one corpus and it's probably best if you only use one. I am using multiple because I am combining them into one model which should speak like a perfect mix of me and my girlfriend.

Taking care of all the hard work

The easiest and one of the best methods for setting up an ebooks bot is to use Mispy's Ebooks Example. Download the zip or clone the git somewhere.

You only really need one file from the archive, bots.rb, but I still copy over the Gemfile and Procfile just for posterity. THE FUTURE GENERATIONS MUST KNOW.

The ebooks example bot is a fully functioning bot that will post automatically, reply automatically, and fav tweets on it's timeline. It can be modified to make something more to your liking, but more importantly it's a great starting point for newbies and advanced users alike.

When you've copied the files over, open up the Gemfile in your favourite editor. I use nano for this because I'm a dork. Modify it to contain whatever version of ruby and twitter_ebooks you're using. The current ebooks_example has an issue with using 3.0.x+, it really REALLY wants to install 3.0.0. I've set it up for 3.0.8 requiring ruby 2.2.0.

Once you've modified and saved the Gemfile run this command.

After you've run this, it's time to consume the corpus.

Once the model is created, it's time to open the bots.rb in your favourite editor. I'm using Atom . 

Fill in the Consumer Key and Consumer Secret from your app and save the bots.rb. 

While you're here change the line

to

The cron line will only post once a day, 3600 will post every hour. You could set to 1800 if you want a half hour timer... or some eccentric time in between.

If you're an advanced user or ambitious, add the line:

between the load_model! and scheduler lines. That will make it post a tweet on start up which is excellent for testing!

Return to your terminal or command line and run this command:

If your auth information is filled in correctly, it will produce a link. Copy and paste that URL in your private browsing session and authorize the app for your bot's account. Paste the PIN into the terminal and press enter. Make sure the account IS correct.

Once the pin is pasted it should produce the final two keys. These will be used to control the ebooks bot account.

Return to the bots.rb and go to the very bottom to make the final changes.

Add the two oauth keys here, change the name of the bot which is enclosed in Clonebot.new(""), and finally change the name of the model (or Original user).

Once this is done, return to your terminal and give it a shot by tying this command:

If it works the output will look something like this:

If you're only aiming to run the bot locally, this is the last step. However, if you want to have your bot hosted freely, 24/7 365, on the cloud, continue to the next step:

Pushing to Heroku

To do this step you will need to set up heroku and install git. If you have a heroku account already, excellent. If not, head here: https://www.heroku.com/

After you've signed up for heroku you will need to get the toolbelt. It's available for a large number of systems. I've never quite got it to work on BSD but if you're somehow this far on BSD, you're already far more advanced a user than I expect.

https://toolbelt.heroku.com/

As I already had heroku installed, I have no screenshots for you. Sorry, but you're on your own.

Once the toolbelt is set up, return to your terminal and type:

Enter your login credentials and you should be good to go.

As for git, most linux distributions have it by default and I'm not really sure what I did to get it working on OS X, presumably homebrew. Git is an absolute must for this next step so it's imperative to get it working.

To get your bot ready for heroku type these commands:

Once the push has succeeded, scale the dyno and the bot should be up and running.

If the bot isn't tweeting or replying after a minute or so, run this command and start debugging:

At this point you should have a fully functioning and automated twitter_ebooks bot. Of course, there are about a million small things that could mess up this process. Most are easily solvable with a google search, but if you're still having issues contact me and I'll see if I can help in some way.

Enjoy and thanks for reading!