July 18, 2018

My First Sinatra Project

Slowly, but surely, I am working my way through the Flatiron School's curriculum. Though it is taking me a bit longer than I expected, I also have relocated to Vermont, sold a house, and settled into my new home. So, I guess I shouldn't be too hard on myself. Living in Vermont is great and I'm hopeful that I can refocus now on getting my Flatiron work done.

To that end, I have completed my first Sinatra application. Sinatara is a framework for web apps that, along with ActiveRecord, can generate some pretty powerful CRUD based apps. Sinatra provides 'routes', which are basically pathways to content that your app will follow in response to HTTP requests. These routes allow for loading and manipulating data through Ruby methods.

For my Sinatra app, I imagined a survey system that would allow users to both create and take surveys. I made some decisions along the way to limit the scope of what I would create for this project. These limitations were selected so that I could fulfill the requirements of the Flatiron curriculum, but also get the project done without going crazy. I'm still going crazy anyway.

  • The surveys will be arbitraryily limited to five questions and each question will only have four possible answers. This is really just for the interface. The data model is designed to accomodate any number of questions and answers. I just didn't want to have to learn some JavaScript to accomplish an interface for this that made sense.
  • I did not complete the actual "taking the survey" part. Instead, in order to fulfill my requirements, I focused on the "create, edit, and delete" a survey part.

To begin, I though about my database and the relationships between the data. I came up with a model that looks something like this:

Sinatra Project Database Model

The "responses" model was not copmleted as previously indicated. So, there are four databases that I used:

  • users: This is for user information, including name, email, password, etc. I use secure passwords with bcrypt, so the database actually uses password_digest.
  • surveys: A user can have many surveys and a survey belongs to one users. Each user can create surveys, edit or delete their surveys. In theory, they could also take another persons's surveys.
  • questions: A survey can have many questions, and a question belogns to one survey.
  • answers: An answer belongs to a qeustion. But a question can have several answers. These would be presented as multiple choice options if I ever get to the "take a survey" part.

So, with that basic model in mind, I got to building the CRUD functionality, models, routes, etc.

To illustrate some of what I learned, let me go over the user authentication and login procedures:

class SessionController < ApplicationController

  get '/login' do
    if !logged_in?
      erb :'sessions/login'
      @user = User.find_by(email: session[:email])
      redirect :'/'

  get '/registration' do
    if !logged_in?
      erb :'sessions/registration'
      @user = User.find_by(email: session[:email])
      redirect :'/'

  post '/registration' do
     @user = params[:username], firstname: params[:firstname],
                      lastname: params[:lastname], password: params[:password],
                      email: params[:email])
     if !logged_in? && @user.valid?
      session[:email] = params[:email]
      @my_surveys = @user.surveys
      @all_surveys = Survey.all
      redirect '/surveys'
      flash[:message] = ""
      @user.errors.messages.each do |key, message|
        message.each do |text|
        flash[:message] += text + "<br />"
      redirect :'/'

  post '/sessions' do
    login(params[:email], params[:password])
    @user = current_user
    @my_surveys = @user.surveys
    @all_surveys = Survey.all
    redirect '/surveys'

  get '/logout' do
    redirect :'/'


The SessionController manages the user login experience. But three helpers I created in ApplicationController are also relevant:

  helpers do

    def logged_in?

    def login(email, password)
      user = User.find_by(email: email)
      if user && user.authenticate(password)
       session[:email] =
       session[:user_name] = user.firstname + ' ' + user.lastname
       redirect '/login'

    def current_user
      if logged_in?
        User.find_by(email: session[:email])
        redirect '/'


These helper methods assist the login process by providing the following funtionality:

  • logged_in? simply checks the session hash for the existence of an email. Is the session has stores an email, that means the person is logged in and thus this method returns true.
  • login takes an email address and a password as inputs. It then checks to see if that user exists and the password authenticates. Password authentication is handled by bcrypt. If the user is found and authenticated, the session hash is updated. Otherwise, the user is returned to the home page.
  • current_user just returns the current users object if there is one logged in.

Now, looking back at SessionController, there are four Sinatra routes that manage the login process:

get '/login' do: This route checks to see if a user is logged in already. If not, it renders the login form. If they are logged in, the user is redirected to their home page.

get '/registration' do: This route renders the registration form if the user is not logged in already.

post '/registration' do: This route handles the registration process. This is where I used validation and flash messages to provide feedback to the user. The validation flash settings are configured in the User class:

class User < ActiveRecord::Base

  validates :username, presence: { message: "You must provide a username." },
            uniqueness: { message: "Username is already taken." }
  validates :email, presence: { message: "You must provide an email." },
            uniqueness: { message: "That email is alreday taken." }
  validates :password, presence: { message: "Password cannot be blank." }

  has_secure_password validations: false

  has_many :surveys


post '/sessions' do: Here, this route logs the user in and then send them to the '/surveys' route where their surveys will be listed for editing, creating, etc.

get '/logtout' do: Finally, this route clears the session hash, logging the user out.

Ok. This post is long and fairly boring, so I'm going to stop here. The CRUD aspect of this app will have to wait for another blog post. That stuff is handled by the SurveyController class. Fun!

March 6, 2018

Making A CLI in Ruby

For my educational benefit and as part of the process at the Flatiron School, I recently wrote a CLI gem in Ruby. This was a great learning experience and a bit of challenge for a newbie like myself. That said, it was also rewarding and I have gained a much better understanding of things Ruby. Mission accomplished.

My idea was simple, since this was my first effort. So, I decided to make a CLI that searches for a list of doctors in a particular zipcode and then allow the user to get more details about a particular doctor. I cleverly named this app "doctor_finder." Why? Because it finds doctors!

At first, I thought I would use one of the large, well-known sites, like WebMD or HealthGrades. I also looked at insurance companies websites and government websites. All of them posed challenges that I could not overcome. The commercial sites blocked scrapers, and I wasn't savvy enough to get around their blocks. Other sites required the user to fill out forms that I could not easily get past in an automated way.

I finally settled on Zocdoc. This website met my needs and allowed me to search for a doctor by zipcode by using a customized URL. It returned 20 doctors to me, that I could parse with Nokogiri.

But first things first. The first step in this whole process was setting up properly for a Ruby Gem. This process is made simple thanks to Bundler. Bundler includes a gem command that creates the directories and basic files you need to get started. It also includes some automation in your gemspec file so you don't have to manually maintain your list of files. Finally, it sets up a git repo for you. Basically it gets you off to a good start:

bundler gem doctor_finder

Bundler will even ask you if you want to include a license and a code of conduct. The basic structure of a gem includes a bin directory for storing executable files, a lib directory for storing the main code, modules and classes, etc, and a spec or test directory for your testing tools. I didn't use any tests for this project, but I'm sure I will in the future.

With that done, the next step is to figure out how you will set up your dependencies and files to get things working. Since I was building a CLI, it made sense to start there. I decided that my CLI would call a CLI object and run a method on that object - and that that CLI object would take care of things from then on. So, to get that going, I needed to create a file in the bin directory called 'doctor_finder.rb' which would serve as my main executable file. So I also needed to chmod +x doctor_finder.rb to make it executable. Then, I added this code to the new file:

#!/usr/bin/env ruby
require_relative "../lib/doctor_finder"

So, the key line is the First, I assume a moduel (namespace) called "DoctorFinder" that is actually created by bundler in the versions file. Then I assume an object called CLI, I create a new one, and I call the method call.

So I need to create the CLI class and include a method called call. I'm pasting in the completed object below. This file would be found in the doctor_finder/lib/doctor_finder directory.

# CLI Main file defining the CLI class

class DoctorFinder::CLI

  def call
    puts "\nWelcome to Hooper's Doctor Finder."
    puts "\nWith HDF you can retrieve a list of doctors by zipcode and then
          get more details about a particular doctor on that list. It's easy!"

  def get_zipcode
    # Gets a valid zip code from the user
    zip = ""
    while !iszipcode?(zip)
      puts "\nPlease enter a valid zipcode:"
      zip = gets.chomp[0..4]

  def iszipcode?(zipcode)
    # Provides a basic level of validation for user input of zipcode.
    if zipcode.length == 5 && zipcode.scan(/\D/).empty?

  def show_list(zipcode)
    # Calls scraper and prints a list of doctors based on the zip code entered by the user.
    docs = DoctorFinder::Scraper.scrape_by_zipcode(zipcode)
    docs.each.with_index(1) do |doc, i|
      puts "#{i}. #{} - #{doc.speciality} - #{}, #{doc.state} #{}"

  def get_choice_from_list
    # Gets a valid choice from the list of Doctors.

    choice = nil

    while choice != "exit" && choice != "q"
      puts "\n[1..#{DoctorFinder::Doctor.all.length}] Select Doctor |
            [zip] Start over with new zipcode | [exit] To quit"
      choice = gets.chomp
      if choice.to_i > 0 && choice.to_i < DoctorFinder::Doctor.all.length+1
        doc = DoctorFinder::Scraper.scrape_for_details(DoctorFinder::Doctor.all[choice.to_i-1])
        puts "======================================\n"
        puts doc.street
        puts + ', ' + doc.state + ' ' +
        puts "--------------------------------------\n"
        puts "Areas of Specialty:"
        puts doc.areas
        puts doc.details
      elsif choice == "zip"

  def farewell
    # Tells the user goodbye.
    puts "\n\nThank you for using Hooper's Doctor Finder.
          This was an educational experiment, and I learned a lot.
          At first it seemed hard, but then it got easier.\n\nSee you next time.\n\n\n\n"

Let me explain the purpose each method:

call: This is the initial method that is called from the executbable itself. All it does is print some introductory content and then call other methods. First I want to get a list of doctors based on a zipcode the user enters. So, I call a function list_doctors and I pass it get_zipcode. After that, I want to run the main menu loop, and then, when the user is done, say farewell.

get_zipcode: This method simply asks the user for a zipcode and then returns that zipcode.

iszipcode?: This method runs some very basic validation on the user input. The good news is that Zocdoc is very tolerant of user input - so even if they don't enter a proper zipcode Zocdoc will do its best to return a list of doctors to us.

list_doctors: Prints out a list of doctors based on a zipcode passed to it. In order to accopmlish this, it makes calls to two other classes, Scraper and Doctor. I'll show you those below.

get_choice_from_list: This is my poorly named main menu loop. Until the user enters valid input, it shows a list of doctors, pulled from the Doctor object. If they choose a valid doctor, it prints that doctors details.

farewell: Says goodbye when the user exits.

Next, lets take a look at the Doctor object. All this class does is store data for a doctor and it maintains an array of all the doctors:

# The Doctor class

class DoctorFinder::Doctor

attr_accessor :name, :url, :speciality, :street, :city, :state, :zip, :details, :areas

  @@all = []

  def initialize
    @@all << self

  def self.all

  def self.clear
    @@all = []

Not much going on here. But if this was part of a larger application, having the Doctor class as a separate class might prove useful. One thing to note - I made a clear method to empty out the array of doctors so I could do a new search in a new zipcode without creating another instance. For the rest of the app, the action is in the Scraper class:

# The Scraper class

class DoctorFinder::Scraper

  BASE_URL = ""

  def self.scrape_by_zipcode(zipcode)
    html = Nokogiri::HTML(open(
    slice = html.css('.js-prof-row-container')
    slice.each do |doctor| # will go through the HTML and create new doctor instances
      doc = = doctor.css('.js-profile-link').text.strip.gsub("\n", ' ').squeeze(' ')
      doc.speciality = doctor.css('.ch-prof-row-speciality').text.strip
      doc.url = BASE_URL + doctor.css('.js-profile-link')[0]['href']
      address = doctor.css('.js-search-prof-row-address').text.strip
      doc.street = address.slice(/^\d+[ ][\w+[ ]]+/)
      # To format the text correctly, had to use some regex = address[/[ ][ ]+[\w+[.]*[ ]]*[,]/].strip.chop
      doc.state = address[/[A-Z][A-Z]/] = address[/\d{5}/]

  def self.scrape_for_details(doctor)
    html = Nokogiri::HTML(open(doctor.url))
    doctor.details = html.css('.profile-professional-statement').text.squeeze(' ')
    if doctor.details.strip == ""
      doctor.details = "No further details were available."
    doctor.areas = html.css('li.specialty').text.squeeze(" ").gsub("\r\n \r\n ", "\r\n").lstrip

The Scraper class has two methods and one constant. The BASE_URL constant was just a helpful way to desingate the web site I was scraping. The other methods:

scrape_by_zipcode(zipcode) takes a zipcode and scapes Zocdocs for a list of doctors based on that zipcode. I encode the zipcode in a search string nad usethat from my open-uri request. I slice a chunk of html out of the whole site to narrow in on just the doctor info. Then I iterate through the array from Nokogiri tossing the data I want into a new Doctor instance. I use a number of regular expressions and string methods to get only the data I want in the object. I return the array of all the doctors.

scrape_for_details(doctor) takes a doctor and fills in more data about that doctor by scraping the doctor's detial page. I use the url I scraped before when I constructed the doctor list and pull down the details. Then, like before, I use text methods and regular expressions to get the data I want in the right place. I return the doctor, but now with more details.

So, that's basically how this things works. As I said at the begninng, I learned a lot playing with this. Now, onward!

February 16, 2018

Object Orientation and Ruby

Ruby is an object oriented programming language.

When I was in college in the early 90s, OO was a peripheral subject. We learned about object oriented programming, but it was reagarded as a side show. There were special cases and certain applications that benefited from OO, but it wasn't the main event. We were focused on procedural programming, in highly structured languages like Pascal or lower level languages where we could get real speed advantages like C.

OO has now become a fundamental approach in most high level programming languages. Ruby was developed, in part, to integrate object orientation throuhgout the programming language. Everything in Ruby is an object, for better and worse.

Even a number.

So you can write code like:


Which takes the object "1", which is an instance of the Fixnum class, and runs the "+" method on it which adds the argument, in this case, "2". The result is "3".

Now, that's kinda funny, so you can also just write:

1 + 2

and all works just fine. I suppose that is a convenience when dealing with numbers, but it is good to always remember that everything is implemented as an object.

Generally speaking, objects contain both their data structures and their code. In the parlance, the data are called "attributes" or "properties" and the code are called "methods." On occassion, it is useful to reference the instance of the object in a method, in order to operate on the instance itself. For that, you use the keyword self. Generally, self refers to the current object. But self can also refer to the current class. Ruby uses context to distinguish what exactly self means. So, for example:

class Person
    @name = ''
    @@count = 0

    def name

    def initialize

    def self.count
        @@count += 1

In the example above, refers to the instance variable @name. In that case, self refers to the instance created by initialize. However, in the method declaration def self.count self now refers to the class since using self in a method declaration makes the method a class method.

This is both confusing, and intuitive. It's confusing because the word 'self' is being used to refer to two different things. But it is also intutive, so long as you consider the context. When you are defining a class, at the top level of that class, self should refer to the class itself, like in the method example. But, within an instance method, self naturally refers to the instance of the class, since the method will be used and 'owned' by instances.

Well, ok, maybe it isn't super intutive. And there is a good chance that I got the details a bit wrong here....But that's the idea.

January 2, 2018

Reclaiming the Internet (and embracing the well-intended flame)

When I was in collge, back in the early 90s, I was first introdcued to the internet. I had an account on my school's mainframe, ostensibly for academic reasons, and I had my first ever email address. Back then, no one had email addresses. I could only email my fellow nerd students and a few professors. Soon, I discovered Usenet, and I followed a number of newsgroups religisouly, especially the one's that commented on beer in a world before micro-brews.

In the early 2000s the internet began to mature. Soon, websites were a thing. As were companies trying to sell you stuff on websites. With the Web 2.0 evolution, websites became dynamic and databse driven and real commerce could be done. I remember learning the basics of Perl and CGI when that was the number one way to create dynamic web content. It wasn't pretty, but it worked.

My life took a different turn. I went to seminary and became a pastor, while the internet continued to change and evlove, largley driven by economic forces. But I remember a few moments in the early 90s when the internet was about something else. It was about connecting with people, sharing information for the good of all, and, yes occassionally flaming people for not RTFM.

That said, whenever I was duly flamed, I was spurred to learn more. When the internet was not as mean-spirited a place as it is now, a flame was a helpful slap to encourage you to dig deeper, do your homework, be a contributor and not just a consumer. The internet, back then, expected me to have something to offer - and because of that expectation, usually, there was something to learn, something to gain, something to share.

Now, between Facebook and Twitter, the internet has become about shouting at people. Or narcissistically sharing the highlight reel of your life. Or about tempting our consumerist cravings until they explode in credit card debt.

I wonder if it could be different. Hosting this blog, on a server I pay for, with a set of open source tools that empower me to make the blog, all of which is free from anyone's ownership or commercial ambition, is a reminder of what the internet can allow. The internet can allow us to freely express ourselves, to engage each other with ideas, to share what we are up to. And it can be a profound community that challenges us to be more thoughtful, more self-critical, and more engaged in meaningful, helpful ways.

Recently, I signed up for, a new service that hopes to provide a more open, more accesible alternative to Twitter and Facebook. By providing an accessible framework within which people can share short posts of content, enables the internet to resemeble its old self. If Manton ('s founder) can find a way to keep free of the commercial interests that have diminished the effectiveness of tools like Twitter and Facebook, and if the community continues to evolve in an intentional way, with strong and clear community guidelines, the future could be bright for and by extension, the internet.

(Usenet began to decline when it's servers were overwhelmed by people uploading pornographic images and pirated software binaries. I'm not judging, but good community guidelines are essential to ensure that any network or community lives up to it's best ideals and original vision.)

The end of net neutrailty threatens new ventures like If it turns out that the only real purpose for the internet is to make some people rich, services like and the old internet have no hope. Eventually, it might become necesssary for an entirely new network to be built - a public network that benefits all people. For now, let's hope that enough of us care enough about the internet to realize it's potential to really, deeply, meaningfully improve people's lives.

November 12, 2017

Getting Stuck...and Unstuck

Getting stuck is part of life. It's also part of programming.

As a fairly novice coder, I find the primary reason I get stuck is because of a pesky syntax error. Somewhere I forgot to put a parenthesis, or I forget to properly close a code block. It's easy to miss these things, and the interpreter is unforgiving.

There are a couple of tools and techniques that I find help when I get really stuck.

  • Carefully read the specs. At Flatiron, many of our labs are coupled with a set of specifications utilizing the RSpec tool set. RSpec tells you exactly what it expects and what you need to do to satisfy it's requirements. Reading the out from RSpec carefully so that I understand exactly what is wanted is a good first step.
  • Examine code character by character. Syntax errors stop execution of code before RSpec executes its tests. Usually an error message will tell you the line number that is presenting the problem, but sometimes the actual error is elsewhere. When I can't figure out where my syntax error is, I slow down....way down... and examine my code line by line. And even then, sometimes it's not enough. I have to examine my code character by character to make sure each and every token is exactly what I want.
  • Find examples and compare. Sometimes I'm just way off. If I'm not sure how my code should be structured or I'm not sure how to use a certain method or library, I sneak online and look for examples or documentation. The online world is amazingly full of helpful stuff - so searching online is usually quite helpful. I also have a handy little app on my Mac that I highly recommend. Dash for MacOS downloads online documentation to your local storage and allows you to easily search for whatever you are looking for. It even has an iOS companion app that will display your search results on your iOS device. I open up my iPad next to my keyboard and search for help all the time now. The Mac app will download almost any open source documentation you can think of - including libraries and tools like Vim and Sass. And it updates the content for you from time to time so you have correct and current info.

These tricks help me move beyond a stuck point. But you know what helps when I'm really stuck? I step away from the desk and take a nice long walk, watch some TV, or (don't tell) play a video game. Taking a break and doing something unrelated really helps. When I return to the desk, I usually find the problem much more quickly than I would have otherwise, thanks to having a refreshed perspective.