Ruby, etc

Learn all the things

He who enjoys doing and enjoys what he has done is happy. - Fortune Cookie

Two Cool Tools for Clojure Development

I’ve recently used a couple tools that I want to write about because I think they are pretty useful.

1. Eastwood

Eastwood is a linter, which will check your syntax to see if it is valid. BTW I have actually made this mistake for real! The github page lists all the things it checks for and a lot of how to configure eastwood.

To get started, Install into your lein profiles.clj:

1
{:user {:plugins [[jonase/eastwood "0.2.1"]]}}

My function, which by the way, is awesome:

1
2
3
(defn my-awesome-function [x y z]
"My function is awesome"
(+ x y z))

Running lein eastwood in the project shows me:

1
2
3
4
5
6
7
8
9
10
11
lein eastwood
== Eastwood 0.2.1 Clojure 1.7.0 JVM 1.8.0_60
Directories scanned for source files:
src test
== Linting blog-post.core ==
Entering directory `/Users/nolastowe/practicing/clojure/blog-post'
src/blog_post/core.clj:5:7: misplaced-docstrings: Possibly misplaced docstring, my-awesome-function
src/blog_post/core.clj:5:1: unused-ret-vals: Constant value is discarded: "My function is awesome"
== Linting blog-post.core-test ==
== Warnings: 2 (not including reflection warnings)  Exceptions thrown: 0
Subprocess failed

Doh. The docstring is in the wrong place, correcting that and eastwood is happy. That is an easy mistake to make!

1
2
3
4
5
6
7
lein eastwood
== Eastwood 0.2.1 Clojure 1.7.0 JVM 1.8.0_60
Directories scanned for source files:
src test
== Linting blog-post.core ==
== Linting blog-post.core-test ==
== Warnings: 0 (not including reflection warnings)  Exceptions thrown: 0

I’ve used eastwood a lot when I was developing amazon lambda functions, when I couldn’t see the result till I’ve uploaded to amazon. At first was frustrated only after all that to find an error which required me to do another compile to jar and upload. Now I use it before commit or when i’ve completed a unit of work or when I just can’t figure why something is not working..always a chance I did something stupid with syntax.

2. Criterium

This library is for benchmarking clojure and I was using it to compare different implementations when I was working on adding some string functions to Clojure. Initially I added it to my lein profiles (similar to how i did with eastwood) which works fine for testing in the repl. Ghadi saw me talking about using it in Clojurians Slack and gave me a bash script to start any version of clojure and criterium with a repl:

1
2
3
4
5
6
#!/bin/bash
M2=$HOME/.m2/repository
CLASSPATH=${M2}/criterium/criterium/0.4.3/criterium-0.4.3.jar:src
JOPTS='-Xmx4G -XX:+UseG1GC'
CLOJURE_JAR=$M2/org/clojure/clojure/${CLOJURE_VERSION}/clojure-${CLOJURE_VERSION}.jar
java $JOPTS -cp ${CLASSPATH}:${CLOJURE_JAR} clojure.main

Then save it as repl-version.sh and run with CLOJURE_VERSION=1.7.0 repl-version.sh to open a repl, then you can test away. I see this being useful when you are working on clojure itself or want to compare things in different versions of clojure. I used it when I was working on a patch for Clojure.

One of the things I used it for was figuring wanted to compare char? and instance? which should I use.. this is a true story! I was trying to figure this out one day..

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
user=> (quick-bench (char? \c))
Evaluation count : 39012534 in 6 samples of 6502089 calls.
Execution time mean : 5.188821 ns
Execution time std-deviation : 0.351751 ns
Execution time lower quantile : 4.755146 ns ( 2.5%)
Execution time upper quantile : 5.544008 ns (97.5%)
Overhead used : 10.020671 ns

user=> (quick-bench (instance? Character  \c))
Evaluation count : 51496848 in 6 samples of 8582808 calls.
Execution time mean : 2.510016 ns
Execution time std-deviation : 0.701679 ns
Execution time lower quantile : 1.693617 ns ( 2.5%)
Execution time upper quantile : 3.375066 ns (97.5%)
Overhead used : 9.812462 ns
nil

I wondered, how does char? work? So I looked at the source:

1
2
3
4
5
6
7
8
user=> (source char?)
(def
^{:arglists '([x])
:doc "Return true if x is a Character"
:added "1.0"
:static true}
char? (fn ^:static char? [x] (instance? Character x)))
nil

Ahh, I see, it just checked the instance? .. so I look what that function does:

1
2
3
4
5
6
7
8
user=> (source instance?)
(def
^{:arglists '([^Class c x])
:doc "Evaluates x and tests if it is an instance of the class
c. Returns true or false"
:added "1.0"}
instance? (fn instance? [^Class c x] (. c (isInstance x))))
nil

So of course, instance will be faster than char …. If you know your class of object, it looks like instance? will always be a little faster. But there ya have benchmarking in Clojure with Criterium. :)

When reviewing this post, Ghadi said:

One small note, instance? is intrinsified by the compiler. It turns into a bytecode instruction. Which you can see in this clojure source. It’s faster than a function call.

Book Review - Clojure Applied

Clojure Applied – Written by Alex Miller and Ben Vangrift.

I am a junkie for books and Clojure books are no exception. I have been doing Clojure since Jan 2014 and feel like I kinda got the hang of it but now what? When I saw Clojure Applied “From Practice to Practitioner” was being written I kept a close eye on Pragmatic Programmer’s coming soon list and an eye on twitter. The day I heard the beta book was available I bought it. When I was offered a free copy to do this review, I said thanks but I already have it and I’ll still be happy to write a review. :)

I was not disappointed as I dived into the book, this is just the thing I needed. Here’s is how to do clojure for the real world with real application of clojure’s concepts applied.

This book is in 3 parts:

  • Foundations
  • Applications
  • Practices

Foundations

Chapter 1: Modeling your Domain

This section covers aspects of your application including modeling your domain using maps and record and when to use each of those. Getting your mind around immutable data is one of the hardest parts for me in getting into clojure and the discussion on entities here really helped solidify the concepts. Once your data is modeled you want to do operations, in OOP languages you have polymorphism. In clojure there are multi methods and protocols. Here are examples of each and when you would use them.

Chapter 2: Collecting and organizing your data.

This chapter talks about collections and which one to use and how to write a custom sorting function. Then it goes on to talk about updating data - wait I thought data was immutable in clojure? It is, so when you update a collection it returns a new copy with the data updated. Its typical of mutable functions to end in ! (danger Will Robinson!), This chapter closes with how to define a “new” datatype and what is involved there.

Chapter 3: Processing Sequential Data

This chapter starts with explaining how sequences are an abstraction connecting collections and the functions that act on those collection. This is the really beauty of clojure and at first mind blowing then once it sinks in, you can imagine how you could have thought differently about the concept. Reducing, Filtering, Selection, Sorting, Removing Duplicates and grouping are all operations you will want to do on your collection once you have it.

Applications

Chapter 4: State, Identity, and Change

Change, if our applications couldn’t change they would be boring, probably even useless. If you need more than one change at a time, you should use a transaction. Different data types (atom, ref, etc) have different functions to update and this goes over each of them in detail.

Chapter 5: Use your cores

This chapter talks about threads and introduces the concept of agents and promises to transfer a value from one state to the other. The talk about using reducers to process data in parallel and the performance considerations. Ending with with discussion about thinking in process using channels in core.async to create a sort of a pipeline for processing data. More meat to this chapter than I can summarize here.

Chapter 6: Creating Components

One of my favorite chapters. Now that you have your structure and processing down, how to combine it into something we can put in a package with a nice bow? There are a few ways to organize this and the first is a namespace. You want to use one or more for your code so it is easily read and comprehended. The chapter goes over some common namespaces used in project and what might go into those. You can connect your components with channels and structure their lifecycle with a record and functions to make start and stop. Leading to the next chapter…

Chapter 7: Compose your Application

Composing your application! Early in the chapter it starts talking about Stuart Sierra’s library name precisely that. Component. It’s like the previous chapter laid the foundation of why, and now this is how. Then putting those into a system and using profiles and configs to make your code operate in different environments (dev, staging etc).

Practice

Chapter 8: Testing Clojure

Ok maybe this is my favorite chapter. My love of testing has been equated with “sickness” for loving tests so much. This chapter kicks off with repl-based testing, by developing your function and calling it in the repl to work out the inputs/outputs. Then moving on to example based testing which I know as unit testing. The new part to me is in Clojure are “are” test function which can easily test a lot of short tests. Testing exceptions is also a thing which is mentioned briefly. Then some other features of clojure test are fixtures, a handy way to provide baseline data for testing. The section ends with a mention of ‘Expections’ test library which takes a different approach than ‘clojure.test’. This chapter ends with talking about property based testing and generators, which if you don’t know about them will blow your mind and leaving you wondering why we haven’t been testing this way all along, :)

Chapter 9: Formatting Data

This chapter seems shorter compared to the other chapters, talking about edn, son and transit. The first two I get and using them in clojure is awesome. But Transit, the authors follow the explanation with a concrete example, I think I get it but I will need to use it in a project for it to really sink in.

Chapter 10: Getting out the Door

This is where it talks about publishing your code and deployment. A few things to think about when publishing your code is to choose the collaboration platform (Github, Bitbucket etc), the contributing agreements and licensing and the minimum files you should have in your repo. When your code is ready to be published as a library most people use Clojars, Maven Central Repository or a self hosted Maven repo. The last part of this chapter is deploying your application to heroku or provisioning your own server by running a jar or to deploying to an application server.

Appendix

This contains two sections: Roots which talks about some of the concepts and Thinking in Clojure which describes the mindset of how clojure is written and what I think community strives to be like.

Overall this is a fantastic book and I’ve tried to summarize the key points to help you make the decision if this book would help you in your clojure career. I bet it will! :) Go Buy it!

Enumerable: Any?

This is about the handy any? method in Enumerable

The documentation says

Passes each element of the collection to the given block. The method returns true if the block ever returns a value other than false or nil. If the block is not given, Ruby adds an implicit block of { |obj| obj } that will cause any? to return true if at least one of the collection members is not false or nil.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
class Taco
  attr_accessor :meat

  def initialize(meat)
    @meat = meat
  end

  def beef?
    @meat == :beef
  end

  def chicken?
    @meat == :chicken
  end

end

Lets test it out

1
2
3
4
5
6
7
8
irb(main):018:0> taco = Taco.new(:beef)
=> #<Taco:0x007fa374070278 @meat=:beef>

irb(main)> taco.beef?
=> true

irb(main)> taco.chicken?
=> false

Lets create two instances

1
2
3
tacos = []
tacos.push(Taco.new(:beef))
tacos.push(Taco.new(:chicken))

Now since the array has enumerable and we made some handy methods to test the meat of our tacos we can easily check our array to see if we have any chicken tacos:

1
tacos.any? { |taco| taco.chicken? }

If all the elements of the array are booleans as in this case we don’t need the block version:

1
2
3
4
5
irb(main)> possible_chicken_tacos = [tacos[0].chicken?, tacos[1].chicken?]
=> [true, false]

irb(main)> possible_chicken_tacos.any?
=> true

Yep, today was definately a chicken taco day. :)

Simple API Backend for Development With Atoms

When playing with liberator I wished I had a simple way to skip over having a database for playing around with data and I also wanted to write about Liberator but not have to get bogged down with Database stuff. I wrote a simple API backend that uses atoms and I’ll write about this first, and this will be followed with some posts on liberator and what is the simplest setup and gradually building on.

As you may know, Atoms are a way to maintain state, a placeholder for changeable data. In my simple-api I defined two atoms, one for the data, an empty map and another for the auto_increment number.

models.clj
1
2
(defonce id-seq (atom 0))
(defonce recipes (atom (array-map)))

Since I only want to define them once (and they will reset everytime this file is reloaded) I used defonce. Once I run that, I can see the values:

repl
1
2
3
4
simple-api.models> @recipes
{}
simple-api.models> @id-seq
0

The @ is a reader macro that de-references the atom. Now we need a function to add a new recipe. For fun I used Prismatic Schema which I wrote about here.

models.clj
1
2
3
4
5
6
(defn add! [new-recipe]
  (let [id (swap! id-seq inc)
        new-recipe-with-id (assoc new-recipe :id id)
        recipe (coerce! Recipe new-recipe-with-id)]
    (swap! recipes assoc id recipe)
      recipe))

Here’s what each line of the function is doing:

1: Incoming map of recipe data comes in as new-recipe.

2: Change the value of the id-seq atom using swap!. Swap! has always felt a bit strange, you have to pass in a function to change the value… you can’t just pass in say 5 (5 is not a function..derp, to just replace the value use reset!). In this case inc works fine and is what we need.

3: Create a new-recipe-with-id by adding in the id as a key/value pair by using assoc.

4: Take the new-recipe-with-id and use Coerce! returns a validated data structure or throws an exception if it is not matching structure and type.

5: Finally take the recipe and add to our recipes atom as a key/value pair of id/recipe.

6: Return the recipe.

Lets see it in action:

repl
1
2
3
4
5
6
7
8
9
10
11
simple-api.models> (add! {:name "All American Beef Taco" :url "http://www.foodnetwork.com/recipes/alton-brown/all-american-beef-taco-recipe.html" :source {:name "Food\
Network" :url "http://www.foodnetwork.com"}})
{:source {:url "http://www.foodnetwork.com", :name "FoodNetwork"}, :name "All American Beef Taco", :url "http://www.foodnetwork.com/recipes/alton-brown/all-american-b\
eef-taco-recipe.html", :id 1}

simple-api.models> @recipes
{1 {:source {:url "http://www.foodnetwork.com", :name "FoodNetwork"}, :name "All American Beef Taco", :url "http://www.foodnetwork.com/recipes/alton-brown/all-america\
n-beef-taco-recipe.html", :id 1}}

simple-api.models> @id-seq
1

We add a recipe with all valid data. Then examine the two atoms: @recipes and @id-seq and both are correct.

Lets add another recipe:

repl
1
2
3
4
5
6
7
8
9
10
11
12
simple-api.models> (add! {:name "Mexican Grilled corn" :url "http://www.foodnetwork.com/recipes/tyler-florence/mexican-grilled-corn-recipe.html" :source {:name "FoodN\
etwork" :url "http://www.foodnetwork.com"}})
{:source {:url "http://www.foodnetwork.com", :name "FoodNetwork"}, :name "Mexican Grilled corn", :url "http://www.foodnetwork.com/recipes/tyler-florence/mexican-grill\
ed-corn-recipe.html", :id 2}

simple-api.models> @recipes
{2 {:source {:url "http://www.foodnetwork.com", :name "FoodNetwork"}, :name "Mexican Grilled corn", :url "http://www.foodnetwork.com/recipes/tyler-florence/mexican-gr\
illed-corn-recipe.html", :id 2}, 1 {:source {:url "http://www.foodnetwork.com", :name "FoodNetwork"}, :name "All American Beef Taco", :url "http://www.foodnetwork.com\
/recipes/alton-brown/all-american-beef-taco-recipe.html", :id 1}}

simple-api.models> @id-seq
2

Here’s the tests I wrote for this function:

models_test
1
2
3
4
5
6
7
8
9
10
11
(use-fixtures :each
  (fn [tests]
    (add! {:name "test" :url "test.com" :source {:name "mom" :url "mom.com"}})
  (tests)))


(deftest test-repository
  (testing "adding recipe"
    (let [recipe {:name "test more" :url "testmore.com" :source {:name "mom" :url "mom.com"}}
          r (add! recipe)]
    (is (= 2 (recipe-count))))))

The test fixture adds one in, and then my test adds another making it count to be equal to 2. I’m not really liking how I wrote this test…but it works. Suggestions?

BTW, here’s the the recipe-count function:

models.clj
1
2
(defn recipe-count []
  (count @recipes))

Now lets write a function to return a vector of recipe maps:

models.clj
1
2
3
(defn get-recipes [] (-> @recipes
                         vals
                         reverse))

In the code I found as inspiration for this the author doesn’t use the @ but passes it to deref … I think I like using the spiral as it results in shorter code. Any reason why one is better than the other?

Now a function to return just one recipe by id:

1
(defn get-recipe [id] (@recipes id))

Since the key is just the id it is a simple task to look up by id.

Now, the function to update a recipe:

1
2
3
4
(defn update! [recipe]
  (let [recipe (coerce! Recipe recipe)]
    (swap! recipes assoc (:id recipe) recipe)
    (get-recipe (:id recipe))))

Swap here just takes the recipe given and puts it in place according to id. Not too complicated here.

Finally, deleting a recipe by id:

1
(defn delete! [id] (swap! recipes dissoc id) nil)

Disassociate the value at id and return nil.

I was pretty happy with how these functions turned out, and I could easily replace them with korma or honeysql when I was ready. I debated calling get-recipe get! and get-recipes get-all! but since those are not desructive like add, update, delete I thought it was fine to leave them.

I’m still learning best and idomatic clojure, so please speak up if you have suggestions! I will be using this simple-api in future blog post as I start to write about liberator and see my tests for the rest of the functions.

Exploring Arrays Part 6

Continuing on through the array methods… now using ruby 2.2.0preview1 :)

eql?

eql?(other) → true or false
Returns true if self and other are the same object, or are both arrays with the same content (according to Object#eql?).

Lets try it on structs first. A struct is an easy way to make an object in ruby. First I define a struct of a User with two attributes:

1
2
irb(main):001:0>  User = Struct.new(:name, :age)
=> User

Ok now lets create some objects

1
2
3
4
5
6
7
8
irb(main):002:0> bob = User.new("Bob", 35)
=> #<struct User name="Bob", age=35>

irb(main):005:0> bob2 = User.new("Bob", 35)
=> #<struct User name="Bob", age=35>

irb(main):006:0> bob.eql?(bob2)
=> true

Ok we have two objects, same data name and age and eql? returns true.

1
2
3
4
irb(main):007:0> bob3 = User.new("Bob", 32)
=> #<struct User name="Bob", age=32>
irb(main):008:0> bob.eql?(bob3)
=> false

Making a third object with different data returns false as advertised. Lets look at the object_ids for each of these objects:

1
2
3
4
5
6
irb(main):011:0> bob.object_id
=> 70351123976940
irb(main):009:0> bob2.object_id
=> 70351128238140
irb(main):010:0> bob3.object_id
=> 70351124267600

I was curious if the items containing the same values had the same id (probably not..) and it looks like it is not the same object. It looks like the eql? method does compare the values in the attributes of the objects. Cool!

Let’s see what else eql? works on … How about strings:

1
2
3
4
5
6
7
8
9
10
11
irb(main):012:0> s = "what up internet"
=> "what up internet"

irb(main):013:0> s.eql?("blah")
=> false

irb(main):014:0> s.eql?("what up internet")
=> true

irb(main):015:0> s == "what up internet"
=> true

Looks like you can use eql? in place of == if you are doing string equality. Not sure I would do it.. it does look kinda nice but also kind of like java :) Probably I would just use ==. But now I know that if I see it used that way it works as it appears.

Lets try arrays

1
2
3
4
5
6
7
8
9
10
11
irb(main):018:0> colors = %w(red green blue)
=> ["red", "green", "blue"]

irb(main):019:0> colors.eql?(["red", "green", "blue"])
=> true

irb(main):020:0> colors.eql?(["red"])
=> false

irb(main):022:0> colors.eql?(["blue", "red", "green"])
=> false

It works, if the array is an exact match, values and order. That leaves numbers which is pretty obvious, only exact matches since numbers are objects too!

1
"1".eql?(1)

Compare to javascript

1
2
> 1 == "1"
true

:) It’s kinda funny.

fetch

fetch(index) → obj
fetch(index, default) → obj
fetch(index) { |index| block } → obj

I’ve used this method with hashes..ALOT. But looks like it works as advertised. The only reason I can think of needing to use a block to get the default is if maybe I needed to retrieve it from a file or webservice.

1
2
3
4
5
6
7
8
9
irb(main):001:0> colors = %w(red white blue)
=> ["red", "white", "blue"]

irb(main):002:0> colors.fetch(0)
=> "red"

irb(main):003:0> colors.fetch(4, :IDUNNO)
=> :IDUNNO
irb(main):004:0>

Interesting though, if you use a negative number it counts from the end.

1
2
3
4
5
irb(main):004:0> colors.fetch(-1)
=> "blue"

irb(main):005:0> colors.fetch(-2)
=> "white"

Another way I have set a default with getting values is to use the || (pipe pipe) operator, let me try it here:

1
2
3
4
5
6
7
8
irb(main):006:0> colors.fetch(4) || :IDUNNO
IndexError: index 4 outside of array bounds: -3...3
from (irb):6:in `fetch'
from (irb):6
from /Users/nolastowe/.rbenv/versions/2.2.0-preview1/bin/irb:11:in `<main>'

irb(main):007:0> colors[4] || :IDUNNO
=> :IDUNNO

Yikes, so if you want to use fetch and set a default you better pass it as the second parameter :) Alternatively, you could check to see if a value exists and return or default with ||. Probably that is what I would use for an array, but now that I think of it, fetch does make it read better and the default is handy. I should change my thinking on this!

fill

fill(obj) → ary
fill(obj, start [, length]) → ary
fill(obj, range ) → ary
fill { |index| block } → ary
fill(start [, length] ) { |index| block } → ary
fill(range) { |index| block } → ary

This method will set the values in an array to a value. It can take a value to set all to or it can take both a value and index(start end) or range. It can set to a value or via a block. Here’s my experimentation.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
irb(main):009:0> weekly_prices = Array.new(7, 49.99)
=> [49.99, 49.99, 49.99, 49.99, 49.99, 49.99, 49.99]

irb(main):012:0> weekly_prices.fill(29.99, 1..6)
=> [49.99, 29.99, 29.99, 29.99, 29.99, 29.99, 29.99]

irb(main):013:0> weekly_prices = Array.new(7, 49.99)
=> [49.99, 49.99, 49.99, 49.99, 49.99, 49.99, 49.99]

irb(main):014:0> weekly_prices.fill(29.99, 1..5)
=> [49.99, 29.99, 29.99, 29.99, 29.99, 29.99, 49.99]

irb(main):015:0> weekly_prices.fill(29.99, 1, 5)
=> [49.99, 29.99, 29.99, 29.99, 29.99, 29.99, 49.99]

irb(main):016:0> weekly_prices.fill(1..5) { 9.99 }
=> [49.99, 9.99, 9.99, 9.99, 9.99, 9.99, 49.99]

irb(main):017:0> weekly_prices.fill(4.99)
=> [4.99, 4.99, 4.99, 4.99, 4.99, 4.99, 4.99]

Probably I would not go looking for such a method on arrays, but I think its a good one to keep in mind. Might be handy to reset an array to default values like for score keeping.

find_index

find_index(obj) → int or nil
find_index { |item| block } → int or nil
find_index → Enumerator

Hmm say you had an array of your books arranged in order of most favorite to favorite..

1
2
3
4
5
irb(main):032:0* favorite_books = ["Eloquent Ruby", "Design Patterns in Ruby", "Growing Rails Applications in Practice"]
=> ["Eloquent Ruby", "Design Patterns in Ruby", "Growing Rails Applications in Practice"]

irb(main):033:0> favorite_books.find_index("Design Patterns in Ruby")
=> 1

You could say “Design Patterns in Ruby” was your second favorite with an index of 1 :)

first

first → obj or nil
first(n) → new_ary

This one is easy, returns the first item in the list or nil if the array is empty.

1
2
3
4
irb(main):032:0* favorite_books = ["Eloquent Ruby", "Design Patterns in Ruby", "Growing Rails Applications in Practice"]
=> ["Eloquent Ruby", "Design Patterns in Ruby", "Growing Rails Applications in Practice"]
irb(main):034:0> favorite_books.first
=> "Eloquent Ruby"

And by the way, “Eloquent Ruby by Russ Olsen” is the bestest book ever about ruby and you should run now and get it.

flatten

flatten → new_ary
flatten(level) → new_ary

I’ve used this one before… let’s try it out:

1
2
3
4
5
irb(main):035:0> ruby_books = ["Eloquent Ruby", "Design Patterns in Ruby", "Growing Rails Applications in Practice"]
=> ["Eloquent Ruby", "Design Patterns in Ruby", "Growing Rails Applications in Practice"]

irb(main):036:0> clojure_books = ["Clojure for the Brave and True", "Clojure Cookbook", "Clojure Applied", "Living Clojure"]
=> ["Clojure for the Brave and True", "Clojure Cookbook", "Clojure Applied", "Living Clojure"]

There are my two arrays containing the names of my favorite books

1
2
irb(main):037:0> favorite_books = [ruby_books, clojure_books]
=> [["Eloquent Ruby", "Design Patterns in Ruby", "Growing Rails Applications in Practice"], ["Clojure for the Brave and True", "Clojure Cookbook", "Clojure Applied", "Living Clojure"]]

Combining both lists into one array results in an array containing two arrays..

1
2
3
4
5
irb(main):038:0> favorite_books.flatten
=> ["Eloquent Ruby", "Design Patterns in Ruby", "Growing Rails Applications in Practice", "Clojure for the Brave and True", "Clojure Cookbook", "Clojure Applied", "Living Clojure"]

irb(main):039:0> favorite_books
=> [["Eloquent Ruby", "Design Patterns in Ruby", "Growing Rails Applications in Practice"], ["Clojure for the Brave and True", "Clojure Cookbook", "Clojure Applied", "Living Clojure"]]

You can see that flatten combines the two arrays and squashes into one array but doesn’t change the actual array. The favorite_books array is still a combination.

However… I did not know about the parameter of level

1
2
3
4
5
6
7
8
9
10
11
irb(main):040:0> one = [1, 2, 3, ['a', 'b']]
=> [1, 2, 3, ["a", "b"]]

irb(main):041:0> two = [5, 6, 7, ['c','d']]
=> [5, 6, 7, ["c", "d"]]

irb(main):042:0> both = [one, two]
=> [[1, 2, 3, ["a", "b"]], [5, 6, 7, ["c", "d"]]]

irb(main):043:0> both.flatten(1)
=> [1, 2, 3, ["a", "b"], 5, 6, 7, ["c", "d"]]

Level allows you to indicate how deep to flatten it, that is cool. I did not know that before now!

flatten!

flatten! → ary
flatten!(level) → ary or nil

Now this, as with all ! (bang) methods, it changes the object instead of only returning the results.

Well, I did learn a few things writing this up and I hope you did too! It really is amazing to find out what you did not know :) Hopefully these tidbits will come in handy in future array manipulations :) :)

Next post on arrays will continue with our journey though arrays and start with Frozen.

Using Lein-try to Learn Prismatic Schema

When I first heard of lein-try on the Cognitect Podcast (and then again in the Clojure Cookbook I thought wow! How cool is that! I tried it with following the example in the readme clj-time to get familar with using it.

First add lein-try to your ~/.lein/profiles plugins vector, run lein deps to get it installed.

1
{:user {:plugins [lein-try "0.4.3"]}}

I wanted to understand Prismatic’s Schema and wanted to try it. I wanted to start simple so I could understand it.

1
lein try prismatic/schema

A repl starts, then require the library and give it an alias

1
(require '[schema.core :as s])

Ok now ready to play!

First we need to define a schema:

1
2
user=> (def Recipe "a recipe" {:name s/Str})
#'user/Recipe

Then we can use validate on a data structure to make sure it matches

1
2
user=> (s/validate Recipe {:name "Chicken and Rice"})
{:name "Chicken and Rice"}

On success we get back the same structure. However, if we pass an invalid structure

1
user=> (s/validate Recipe {:name 42})

Wait a minute. What is this? an actually helpful error message in Clojure?!?!

1
ExceptionInfo Value does not match schema: {:name (not (instance? java.lang.String 42))} schema.core/validate (core.clj:161)

It tells you the keyword of the invalid value and what value was passed in. Nice.

We can validate, but what does check do?

1
2
3
4
user=> (s/check Recipe {:name "Chicken and Rice"})
nil
user=> (s/check Recipe {:name 42})
{:name (not (instance? java.lang.String 42))}

Looks like you’d use check when you are prepared to deal with nil, and validate when a representation of the data structure is needed.

Now lets try a more complex example.

1
2
3
4
user=> (def Source {:name s/Str :url s/Str :year s/Int})
#'user/Source
user=> (s/validate Source {:name "Food Network" :url "http://www.foodnetwork.com" :year 2015})
{:name "Food Network", :year 2015, :url "http://www.foodnetwork.com"}

This example is Source which consists of 3 elements. Lets try leaving one off:

1
2
3
user=> (s/validate Source {:name "Food Network" :url "http://www.foodnetwork.com" })

ExceptionInfo Value does not match schema: {:year missing-required-key} schema.core/validate (core.clj:161)

This makes me think maybe there is a way to indicate an optional value? Reading down farther on the documentation I see how to make keys optional. Lets redefine our Source with year as optional:

1
2
3
4
5
user=> (def Source {:name s/Str :url s/Str (s/optional-key :year) s/Int})
#'user/Source

user=> (s/validate Source {:name "Food Network" :url "http://www.foodnetwork.com" })
{:name "Food Network", :url "http://www.foodnetwork.com"}

Sweet! I also read that since Schemas are just data structures they are composable. Lets add Source to Recipe to make a more complex structure.

1
2
3
4
5
6
7
user=> (def Recipe "a recipe with source" {:name s/Str :source Source})
#'user/Recipe
user=> Recipe
{:name java.lang.String, :source {:name java.lang.String, :url java.lang.String, #schema.core.OptionalKey{:k :year} Int}}

user=> (s/validate Recipe {:name "Chicken and Rice" :source {:name "FoodNetwork" :url "www.foodnetwork.com"}})
{:name "Chicken and Rice", :source {:name "FoodNetwork", :url "www.foodnetwork.com"}}

Nice!! Since I’m a ruby developer by day, I think how could I do this in ruby? Well, in Rails, we have validations on models and once you set the attributes you could call model.valid? which would return boolean. If it’s not valid, it populates a errors attribute on the model with a nested hash of key/error messages. Composing two together is not as straightforward. You might be able to with the accepts_nested_attributes_for on a Recipe model, but I conclude its not going to be as elegant as using Clojure and Schema :)

Using lein-try made it easy to experiement with a library and poke around to practice :)

Testing Routes With RSpec and Minitest

The poor often forgotten routes folder in your tests or specs folder (if you even have one!). People don’t know what goes in them or how to write them. I like to write them because they are pretty easy to write and often a great first-step when it comes to a legacy codebase.

It gives you the chance to get a good look at that routes file and see if there are routes there you really don’t need? Often the resources :whatever is the culprit, the others are either out of date (the controller/action was removed but routes file never updated) .. I wondered why should you only include the routes you need? what harm is there? It make the rake routes file cleaner but are there other reasons to be very specific in routes.rb ?

So I sought the wisdom of twitter:

Peter Harkins @pushcx
Documentation to future coders.
--
Karthik Hariharan @hkarthik 
avoids the "action controller method not found" errors and pushes 
a 404 instead of a 500. Really important with REST APIs.
--
Brad Fults @h3h
FWIW, I highly recommend avoiding `resources` and preferring 
a `get/post/patch` for every route. Much clearer.  
--
Caleb Thompson @calebthompson 
:only 

I got some really good answers!! See all the responses

Avoiding the resources magic is one way to make it very clear:

1
2
3
get "/users", "users#index"
get "/users/:id", "users#show"
post "/users", "users#create"

Verses resources style:

1
resources :users, only: [:index, :show, :create]

Using only is also pretty clear and maybe less messy? This probably come down to personal preference and which way allows you to sleep best at night. Decide with your team so you can be consistant in your routes.rb file.

So how do you test them?

It depends on what test framework you are using. Here’s what I’ve figured it out.

RSpec

1
2
3
4
5
6
7
8
9
10
11
  context "should route to member_sessions" do
    it "should initialize a new session" do
      expect(get: "/member_sessions/new").to route_to(controller: 'member_sessions', action: 'new')
    end
    it "should create a new session" do
      expect(post: "/member_sessions").to route_to(controller: 'member_sessions', action: 'create')
    end
    it "should destroy a session" do
      expect(delete: "/member_sessions/1").to route_to(controller: 'member_sessions', action: 'destroy', id: "1")
    end
  end

This will test that the method, url and params will go to the right controller/action. It seems pretty straight forward right?

Minitest / TestUnit

We have three methods:

  • assert_generates - testing that the provided url, will break down to a controller,action and parameters
  • assert_recognizes - testing that rails will recognize the route, given controller and action, it goes to a certain method and path
  • assert_routing - a combination of the above, almost literally, if you look at the source!

One thing I can’t figure out is how to test the methods along with the routing info like I can with RSpec. We can test in other ways (feature tests), but its not quite as nice as RSpec. I guess that is just a difference in the two frameworks. I found when the same path is common to multiple methods, than assert_generates works and assert_routing does not (and assert_recognizes does not also). For example in the case of Create, Update, Delete and Show they use the same path so I have to use the assert_generates to that the path goes to the right controller.

1
2
3
4
5
6
| method | path          | controller |
| ------ | ------------- | ---------- |
| create | /register/:id |  register  |
| update | /register/:id |  register  |
| delete | /register/:id |  register  |
| show   | /register/:id |  register  |

So once I realized that, I understood (duh) which route testing methods to use. Some might argue these are pointless, but I still think its valuable as it forces you to really think about your routes.

1
2
3
4
5
6
7
8
class RoutingTest < ActionDispatch::IntegrationTest
    it "routes to register index" do
      assert_generates "/register", controller: "register", action: "index"
    end
    it "routes to register show" do
      assert_generates "/register/1", controller: "register", action: "show", id: "1"
    end
  end

These are mostly the same, except for the starting of the test. The testunit style has a class and spec style has a description. The describe string must end in Acceptance Test.

1
2
3
4
5
6
7
8
9
describe "Routing Acceptance Test" do
  it "routes to register index" do
    assert_generates "/register", controller: "register", action: "index"
  end

  it "routes to register show" do
    assert_generates "/register/1", controller: "register", action: "show", id: "1"
  end
end

As I’ve been writing route tests, I’ve increase the security, clarity and interface to my apps. My routes file has become a very clear documentation of the api of my application. I hope that testing routes becomes as important to you as it is for me!

Leveling Up on Rspec

The other day I saw there was a Rspec option in .rspec I didn’t know about and I thought what else do I not know about Rspec? I set myself to dig around in the command line options and source code to see if I could figure out anything else cool. Here are a few of the things I’ve learned:

Require all the time

This is what triggered my curiosity, finding this in .rspec:

1
--require spec_helper

This will add that require to the top of all your test files, and since all your test files have that require at the top, this cleans up the tests files (and you dont forget it to add it). Cool right? This should go in .rspec in the project folder.

Local Options

I knew about ~/.rspec (in your home directory) and .rspec (in the root of project directory), but digging around in the Rspec source there is also .rspec-local which looks to be the place to put personal preferences for the current project. I imagine this might be useful if the team wanted to use a certain formatter and you liked a different one. According to repo this was added 2 years ago, I had no idea! This would be one of the files you would put in your .gitignore.

Dry Run

Want to see how your wording looks with the format documentation (I like to make it read like a happy ending story).

Use this

1
rspec --dry-run -f documentation

You can also add this into a config file, but most likely you’ll just want to run it occasionally so command line is the way to go here.

Fail Fast

Tired of running the test suite and seeing some pass and some fail, but having to wait till all the tests are done? Add --fail-fast to your command line (or .rspec-local) to have the tests stop after the first failure. This is one option I think would be good to keep in local since it might be personal preference.

Run one test

There are many ways to tell Rspec that you want to just run a single test:

Passing --example and the name of the test (it “test name”)

1
rspec --example "should group in words of 3"

Or just enough of the name to make it unique. Also you can use -e instead of –example

1
rspec -e "should group in"

Or you can specify the line number, but be aware this might change as you modify line above this.

1
rspec spec/lib/memory_spec.rb:11

Run Focus Test

An alternative to Run one test is to use the focus. First set it up in the spec_helper to run only the focus: true if it exists anywhere, otherwise it runs all the specs so you can totally leave this in the spec_helper file all the time if you like it!

1
2
# in spec_helper.rb
config.filter_run :focus

Then in your test add focus: true after the name, like so:

1
it "should group in words of 3", focus: true do

I think in future I am going to use focus: true if i am running tests on the command line as I develop them, would make it easier then running the test name or line number because those might change as I develop.

Test Formatters

The default report style for Rspec is fine, I like format –documentation and some time ago I found this one by Tim Pope Fivemat. Add that to your gem file then add in your Rspec config ‘fivemat’

1
2
3
4
add to .rspec
--format Fivemat
or specify on command line
rspec -f Fivemat

Looks like this:

1
2
3
Memory ............
User ......
Address ..

Name of the class, plus a green dot for each passing test. Also for test failures there is less noise so this is a favorite of mine.

Update, added: NyanCat Formatter

I certainly feel like I’ve Leveled Up my Rspec and hope you have too! :) Code on!!

Exploring Arrays Part 5

BTW Going to use Ruby 2.1.2 from here on out

Yes, it has been awhile but (who has time to dwell on the past?) I haven’t forgotten I was going through each array class! My last post said next was delete, so let’s get to it!

delete

delete(obj) → item or nil 
delete(obj) { block } → item or result of block

Ok first one looks pretty straight forward, it returns item if it found it and nil if it doesn’t. Lets try it out:

1
2
3
4
5
6
7
8
9
10
>> colors = %w(red blue green yellow)
=> ["red", "blue", "green", "yellow"]
>> colors.delete("blue")
=> "blue"
>> colors
=> ["red", "green", "yellow"]
>> colors.delete("can't find me")
=> nil
>> colors
=> ["red", "green", "yellow"]

I deleted “blue” and it found it because it returned “blue”. The array is now changed, contains no “blue”. When you try to delete a value which is not there, “can’t find me” it doesn’t throw an exception, just returns nil and does not change the array. What happens if the array is empty? Lets say I made myself a todo list and deleted thing as I finished them:

1
2
3
4
5
6
7
8
9
10
11
12
>> todo = ["get up", "drink coffee", "study ruby"]
=> ["get up", "drink coffee", "study ruby"]
>> todo.delete("get up")
=> "get up"
>> todo.delete("drink coffee")
=> "drink coffee"
>> todo.delete("study ruby")
=> "study ruby"
>> todo.delete("do it again")
=> nil
>> todo
=> []

So nil is the result when the item is not found OR the array is empty. So, don’t depend on the return value too much since it can be nil for multiple reasons. Delete is not a great method for checking off things in your todo list!

delete_at

delete_at(index) → obj or nil

Where as delete would delete from the array at the value, delete_at will delete at the specified index.

1
2
3
4
5
6
>> colors = %w(red orange yellow green blue)
=> ["red", "orange", "yellow", "green", "blue"]
>> colors.delete_at(3)
=> "green"
>> colors
=> ["red", "orange", "yellow", "blue"]

Pretty simple, let see what happens when you try to delete at an invalid index:

1
2
>> colors.delete_at(100)
=> nil

So once again, if its an invalid index you get nil just like if there is an invalid value, you get nil. Lets move on!

delete_if

delete_if { |item| block } → ary
delete_if → Enumerator

So looks like if takes a block that tells us what items will be deleted from the array. Lets delete all color names that are greater then a length of 5 characters.

1
2
3
4
>> colors = %w(red orange yellow green blue)
=> ["red", "orange", "yellow", "green", "blue"]
>> colors.delete_if { |color| color.length > 5 }
> ["red", "green", "blue"]

So it deletes from the array, every element for which the block is true. And its true for values orange, yellow and green. Leaving us with red, green and blue. The docs say its the same as reject! … yay for being like php!

drop

drop(n) → new_ary

So it takes the array and removes the first N elements and returns the rest.

1
2
3
4
?> colors = %w(red orange yellow green blue)
=> ["red", "orange", "yellow", "green", "blue"]
>> colors.drop(2)
=> ["yellow", "green", "blue"]

We drop the red and orange elements, and return the rest which is yellow, green and blue. Pretty straightforward, Ok, next method!

drop_while

drop_while { |arr| block } → new_ary
drop_while → Enumerator
1
2
3
=> ["red", "orange", "yellow", "green", "blue"]
>> colors.drop_while { |color| color.length != 4 }
=> ["blue"]

This one drops all elements for which the block is NIL or False. So if we wanted all the colors whose length was NOT 4, we would only get one element of blue.

each

each { |item| block } → ary click to toggle source
each → Enumerator

This one is easy, we use it all the time..

1
2
3
4
5
6
7
8
>> colors = %w(red orange yellow green blue)
=> ["red", "orange", "yellow", "green", "blue"]
>> colors.each { |color| puts color }
red
orange
yellow
green
blue

each_index

each_index { |index| block } → ary click to toggle source
each_index → Enumerator

Whereas each gives you the value each time, this gives you the index.

1
2
3
4
5
6
7
8
>> colors = %w(red orange yellow green blue)
=> ["red", "orange", "yellow", "green", "blue"]
>> colors.each_index { |color_index| puts color_index }
0
1
2
3
4

empty?

empty? → true or false

This checks if there are any elements in the array, I experimented with nil, empty or whitespace and the only thing that returns true is no elements.

1
2
3
4
5
6
7
8
?> [nil,nil].empty?
=> false
>> ["",""].empty?
=> false
>> [" "," "].empty?
=> false
>> [].empty?
=> true

This seems like a good place to stop, next blog post will start with eql?

Book Review: Application Testing With Capybara

If you know me, you know I love testing. Model tests, library tests (typically when poro objects) are the most awesome to test and I love it. Controller tests, well, are less fun and testing routes is not bad either. But my least favorite are integration tests.

Cucumber tests seemed awesome in theory and I remember early days in Chicago ruby group there was development on rspec story runner which I believed was written by Dan North, later rewritten as Cucumber. Wow!! Project owners could write their own tests!! Right… Or maybe they could read the tests and say yeah that’s what I want. Yes, that was 2007. I’ve toyed with cucumber a few times and in frustration gave up. Even when using capybara, it was hard to get the right syntax and find accurate documentation.

Last fall there was a few talks I watched Capybara and Cucumber, Rspec … One project I work on they use Selenium. Ahh so many choices. How do I know what exactly is the difference between each of them and when do I use what?

I was pretty excited to get a review copy of Application Testing with Capybara … ok, lets give this another shot.

The book starts out with installation, then proceeds into an example of using cucumber to run a test on Youtube, I think thats a great start. Lets focus on writing a test for something we didn’t build and get our setup and syntax down. And here is where we see the Capybara uses Selenium as the driver. I was able to follow along and build a test that does a search on youtube. Cool, you know getting something working while you are learning is a great boost!

The second chapter is about Mastering The API … this is where I got hung up on the most, seeing different tutorials, docs, examples with different versions (most examples don’t tell you what version numbers of Capybara is used!) and it is just frustrating. Capybara can use XPath or CSS selectors to locate elements. The author prefers the CSS selector style because he thinks they are more readable, I agree! You can set your preferred style in your env.rb but you can always override it on a case by case basis.

The book continues with examples of all types of selecting items and filling in forms. One thing that I found helpful is it shows the syntax of the example queries and Rspec matchers

1
2
3
4
5
# example selectors
page.has_selector? '#local_results'

# Rspec matchers
page.should have_selector '#local_results'

This is one of the things that I think always tripped me up, I didn’t know there was two ways to do the same thing and they were both correct, it is just different styles. This is a great resource! The book also talks what driver to use when and how to tag tests that would have client side code. Great content.

Today I wrote some Cucumber tests for a project I am working on and I used this book. I got things setup set up correctly and looked up some syntax. I am happy to report success. If you want to work with Capybara be sure to grab a copy of Application Testing with Capybara :) YAY!