Hands on with Phoenix

In this session, we will introduce Phoenix, which is one of the most popular frameworks for building Elixir-based web applications.

According the official documentation, you can install Phoenix by executing the following command:

 mix archive.install https://github.com/phoenixframework/archives/raw/master/phx_new.ez

Well, there are other methods but the one above seems good enough as a reference to include here.

Once the installation is complete, we would be ready to start working. In a terminal window, run the command

mix phoenix.new takso

The command above resembles somehow the one that used in the past, i.e. mix new, to create a basic Elixir project. This time, however, it is clear that we are creating something else, that is, the skeleton of an application using the Phoenix framework. As before, the name that we are assigning to our project (and hence to our application) is “takso”. Since a phoenix application relies in various libraries, more than the ones used in a basic Elixir project, the tool mix will ask whether we want to install all the additional dependencies, to which we can safely respond with “y” or simply enter, since “yes” is the default.

When the tool mix completes the scaffolding of the application, it will also print out some hints about what to do next. In our case, we would need start by completing the setup of the database. Note that if you configured your postgresql as specified in the email sent via Piazza last week, you can skip this part. (Well, it also makes sense to read this part to learn a little bit about the configuration of our application, so do not be hasty.)

Configuring the database

First, you have to change to the project home directory, i.e. takso in our case. Open your project in your editor or IDE and look for the file config/dev.exs. At the end of that file, you will see the following snippet:

# Configure your database
config :takso, Takso.Repo,
  adapter: Ecto.Adapters.Postgres,
  username: "postgres",
  password: "postgres",
  database: "takso_dev",
  hostname: "localhost",
  pool_size: 10

Clearly, we only have to modify the values for username, password and, why not, database according your own configuration and taste. Let me also note that we are changing this configuration only for the database that we will be using for development purposes. That to say that you will find a similar file, called config/test.exs, to configure the database to be used for testing purposes. In the case of production, the database configuration is located in the file config/prod.secret.exs. Note that this file would not be stored in your git repository by default, to give you more control about who can access such sensitive information.

Once we have completed the configuration of our database, we can proceed by executing command below:

mix ecto.create

Using our configuration settings, the first command will create a database for our project. Well, the first time you run mix ecto.create on a new project, you will see more happening: the tool mix will also launch the compilation of the application. Do not be worried if the compilation process takes long, the compilation is incremental, so any change on your code will be very quickly done in the future. Often, I get some errors in this step just because I forget very often to start my database server before. Hence, it is a good idea to check whether your postgresql server is running if the compilation process complains.

Starting the scaffolded application

The second command we are going to use starts a web server (i.e. the cowbow server that we used in Homework 2) and deploys there our application. Let us give it try, run the following command and then, in a web browser, open the URL http://localhost:4000.

mix phx.server

Well, the web application looks already nice but is still empty. Stop the server with Control+C and let us add a few things.

Before changing anything, I will propose you to set up a local git repository. This will allow us to keep track of the evolution of the application and revert some of our steps that I will propose you to take, in order to understand the structure of a phoenix-based application.

On a terminal, run the following commands:

git init
git add -A
git commit -m "Initial commit"

I am assuming that you have taken the tutorial that I proposed you in the first lecture. If you have not heard about git before nor taken the tutorial, please take a time to go through the tutorial before you continue with this material.

As I mentioned before, our application is still empty. Let us then add one resource to the application. To that end, run the following command:

mix phoenix.gen.html User users name username password

The command above will generate a bunch of code that include: code for database access, web controller, HTML view rendering, among others. If you take a closer look to the feeback provider by mix you will see that at the bottom, it proposes us two things: adding the line resources "/users", UserController to the file web/router.ex and then running the command mix ecto.migrate.

First things first. Open the file web/router.ex and find the block of code where the application configures what is called the browser scope. In the originally generated code, the block should look like:

  scope "/", Takso do
    pipe_through :browser # Use the default browser stack

    get "/", PageController, :index
  end

It is in that scope that we have to add the line. The resulting code should look like the snippet below:

  scope "/", Takso do
    pipe_through :browser # Use the default browser stack

    get "/", PageController, :index
    resources "/users", UserController
  end

Save the changes and then executed the command proposed by mix, that is, mix ecto.migrate. The last command will create a table in the application’s database to store the information about the app’s users. In this moment, I ask you to take this as granted. I will come back later to speak more about databases and migrations.

Let me first speak about the line that we added to file web/routes.ex. To that end, run the command mix phx.routes. If everything is fine, you will see that mix dumps the REST API for our application. The addition of the line resources "users", ... indicated the framework that we are willing to expose a set of operations for interacting with an HTML-based application, using a REST API. At the right hand side of the table, you will see that we are exposing URLs for operations such as creating a new user (identified by atom :create), editing the information of a user (identified by atom :edit), updating the information of a user (identified by atom :update), etc. The function resources is provided by the library Plug.Router that we used in homework 2. In fact, if you analyze the line that comes before the one we added, you will recognize the function get which is followed by the path on a URI. This time, however, we are not including the code to process the request. Instead, the code is written in a way the application will dispatch the request to another component in the application referred to as a web controller, which is implemented in this case within the module UserController.

Before continuing speaking about the code, take a look at effects of the code scaffolding in the application. Run the command mix phx.server and open the application in the browser. This time, however, open the URL http://localhost:4000/users and play a little bit with your application.

Rebuilding the user resource components, step by step

Scaffolding is nice because we get a good start for our application. However, the code generated is too generic such that we will eventually discard most of it, most of the time. Moreover, we lost the opportunity to understand what are the components involved in the support to the resource user. So let us first rewind the tape and start over. Run the command git reset --hardand then git clean -fd. As you should know, the two last commands will discard all the changes made in our project and restore the state of the initial commit.

Note that, at this point, the application’s database has already been updated: we have created a table for resource user and, probably, added some users while playing with the application. Let us revert the changes using the command mix ecto.reset.

From this point on, I will ask you to open another terminal window and run there the command mix phx.server. You will notice that the application is updated every time we add a new component, without the need to restart it again.

Router and Controller modules

Well, we already know that one of the steps is that of adding the line resources "/users", UserController to the file web/routes.ex. Please do it and then, on your web browser, try to open the URL http://localhost:4000/users. Oops! Something went wrong! That is normal, because we discarded the code associated with the resource user. However, you can see that phoenix is providing us with a hint about what is missing and what is the next step to take. In fact, openning a URL in the browser entails a GET operation on the URL. If you check the application routes you will see that a GET on /users triggers the function :index of the moduel UserController. Therefore, create the file web/user_controller.ex and copy there the following snippet:

defmodule Takso.UserController do
  use Takso.Web, :controller

  def index(conn, _params) do
    render conn, "index.html"
  end
end

Please note that the name of module is Takso.UserController and not only UserController. The idea behind is that phoenix promotes the use of a composite name for the modules to provide a hint about the composition of the application. The first part of name of the module is then the same of the application and then we have the name associated with the controller.

Phoenix generates some code that is shared by several components in the application. The second line the code above, i.e. use Takso.Web, :controller, indicates that this module uses the code (and configuration) specified in the application for web controllers.

Now, let us focus on the function index. You will notice that, by convention, the controller methods that are referred from the router receive two parameters: a reference to a Plug.Conn (variable conn) and the parameters used in the request (variable _params). In this very basic form of get operation, we will not use the request parameters. That is why the name of the variable starts with an _.

The last line of the function calls render, an alias for Phoenix.render/2, which passes over the connection conn and the value index.html. This line of code, I would say, is clear: we want to render a page index.html which will eventually show the list of users. If you reload the page in your web browser, however, you will not see such a list but the details of a new error. Well, at least the new error page includes some hints on what to do next.

View component

Interestingly, the error page does not refer to the file index.html but to a certain module Takso.UserView. In phoenix, the notion of view refers to a module that adapts the information sent from the controller to the representation that is required by a specific rendering technology. In this case, we are going to use the adapter provided phoenix out-of-box and leave a more advanced use for the future. Anyway, we need provide at least an empty view module. Therefore, I will ask you to create a file called web/views/user_view.ex and copy there the following snippet:

defmodule Takso.UserView do
  use Takso.Web, :view
end

As you can see, we can recognize there code that we saw previously in the controller. However, this time we specify that we will use the application’s configuration/generic code for a view controller (see use Takso.Web, :view). Save the code and reload the page with URL http://localhost:4000/users on the browser.

HTML index page

We reach the point where we finally see a direct reference to an HTML file. Specifically, it seems that we need to provide a file index.html and that, according to the error page, under the folder web/templates/user.

For the time being, we will just add a very simple HTML file. Copy the following snippet into the file web/templates/user/index.html.eex.

Hello world

I hope you noticed that the file that we added has the suffix html.eex. In fact, the file is not plain HTML but a template that will be used for producing HTML. The extension eex indicates that the templating technology to be used is the one called Embedded Elixir. Well, in this moment we used a simple text, but later we will use HTML with some embedded elixir code.

== TODO: add text for transitioning to databases

Schema component

mix phoenix.gen.model User users name

The command above adds three files to our project: the file web/models/users.ex which holds the code describing the structure of the user information stored in the database and code for validations, among others; a migration file priv/repo/migrations/......create_user.exs which holds a script to maintain the database schema, and the test file which we will not discuss in this session. Note that the name of the migration file uses the current time on you computer for its prefix. This is called the migration timestamp.

Let us start with the migration file. Migration is a concept that several frameworks provide as a tool to ease the creation and maintenance of database schemas in large projects, particularly when multiple developers are working with the same database. The idea is to use timestamped scripts to specify the changes that need to be propagated to the database schema, instead of allowing programmers (or DBAs) to directly change the schemas, e.g. using interactive database consoles. Each migration has a timestamp. Such timestamp is used to decide the order on which the scripts have to be executed. This unique ordering is the key for achieving a consistent view in all the development/testing/production platforms used by the software product.

Open the migration file in your editor. Of course, the code is just an skeleton because this time I decided not to include the attributes associated to a user in the command line during the scaffolding. Please replace the entire content of this file with the following snippet:

defmodule Takso.Repo.Migrations.CreateUser do
  use Ecto.Migration

  def change do
    create table(:users) do
      add :name, :string
      add :username, :string
      add :password, :string

      timestamps()
    end
  end
end

When reading the code, you will notice that we are refering to Ecto, which is Phoenix library for providing database access. In this case, we are using more specifically Ecto.Migration. As you can see, only one function is defined, namely change/0. In fact, every migration should provide two functions: one defining the way the database schema must be modified and one defining a way to undo the modification. In this case, the behavior of the two functions can be inferred from the code specified by change/0: we either create or drop the table.

I believe the code is self-described. We want to create a table with name users (note that we pass the atom :users instead of the string). The table has three columns, each one of type string. The final method call, i.e. timestamps(), instructs Ecto to maintain two timestamps per row: the creation time and the time for the last modification of a given row.

Please do not change the database schema directly. If you need to change a database schema write a migration. To that end, use the command mix phoenix.gen.migration name_of_migration. Try to use a name of migration that clearly describes the goal of it (e.g. add_user_phone_number).

Let us now focus on the file web/models/user.ex. Open it on your editor and replace its content with following code:

defmodule Takso.User do
  use Takso.Web, :model

  schema "users" do
    field :name, :string
    field :username, :string
    field :password, :string
    
    timestamps()
  end

  def changeset(struct, params \\ %{}) do
    struct
    |> cast(params, [:name, :username, :password])
    |> validate_required([:name, :username, :password])
  end
end

If we analyze the code above, we can clearly see two sections. In the first section, we are specifying a database schema. In fact, this section refers to the database schema that we defined in the migration. Note that in the general case, we could change the database technology and hence we would need to change the first section on this module accordingly. For instance, we could define a struct and use an Elixir process to hold a list of that struct to replace the database. Well, supporting advanced queries on the latter data structures would be inefficient, but it was just an example.

The second section of the code is for creating a phoenix data structure called a changeset and to apply validations. To understand what a changeset is, let us first understand that every information about a user that we are bringing from a database into the Elixir’s runtime is to be held on an Elixir struct. Also, everything in Elixir realm is immutable. Henceforth, the changes cannot be applied into the struct but are collected in an auxiliary data structure called a changeset. Moreover, a changeset is also the target of further analysis to verify if the changes are consistent with some business rules. For instance, in the code above we are enforcing that the values of all the attributes is required not be null.

Given the value of a struct, the function cast/3 initializes a changeset. Note that we are providing a list of attributes. This allows us to declare the subset of attributes that we want to disclose. In the last line, we call the function validate_required/2 which, as we mentioned before, checks that the values of some attributes (all of them in this case) are not null or empty string.

To speed up the demonstration, we are going to add some rows into the database using a script. Open the file priv/repo/seeds.exs and replace its content with the following code.

alias Takso.{Repo, User}

[%{name: "Fred Flintstone", username: "fred", password: "parool"},
 %{name: "Barney Rubble", username: "barney", password: "parool"}]
|> Enum.map(fn user_data -> User.changeset(%User{}, user_data) end)
|> Enum.each(fn changeset -> Repo.insert!(changeset) end)

I suppose you understand the code. Just in case, let me briefly explain it. We have a list with two users, each one respresented as a map. We iterate that list and create a changeset with the information for each user. Finally, we call the function Repo.insert!/1 to persist one by one each user.

Well, we have written lots of code. It is time to complete the database setup. If we were just to install the database migration we would use the command mix ecto.migrate (as mix suggested after creating the model). However, this time we also want to run the seeds.exs script to “seed” the database. Hence, we will use the command mix ecto.setup.

Wiring the ends

Let us now connect the database access, the controller and its corresponding list. If you remember, we have implemented the action UserController.index/2, but by now it only forwards the connection and the string index.html to the view. It is here that we connect the database access, a.k.a. repository, with the controller. Replace the current function UserController.index/2 with the following piece of code:

  alias Takso.User

  def index(conn, _params) do
    users = Repo.all(User)
    render(conn, "index.html", users: users)
  end

Well, you can see that I piggy backed the line alias Takso.User. This line is just to shorten my code, because that allows me to write User instead of the long Takso.User. Anyway, you can see that we have a new line of code, where we query the list of all users in our database with the call Repo.all(User) and pass the result as the third parameter for function render. And that makes sense: this function is expected to retrieve the list of users and pass on that list to the view layer to produce an HTML table rendering such list.

If you remember, we had the view component UserView which was simply passing over the information and also a rather simple EEX template (the one that said “Hello world”). Let us then fix the template. To that end, replace the “Hello world” with the following snippet:

<h2>Listing users</h2>

<table class="table">
  <thead>
    <tr>
      <th>Name</th>
      <th>Username</th>
      <th>Password</th>
    </tr>
  </thead>
  <tbody>
<%= for user <- @users do %>
    <tr>
      <td><%= user.name %></td>
      <td><%= user.username %></td>
      <td><%= user.password %></td>
    </tr>
<% end %>
  </tbody>
</table>

The snippet above looks just like a normal HTML file. However, you will see that it contains some elixir code embedded in some places. For instance, the line <%= for user <- @users do %> corresponds to an Elixir comprehension: iterates over the list of users (yes, we are refering to the list of users forwarded from the controller) one by one. Later, it displays each one of the attributes associated with a user, e.g. <%= user.name %>. Do not wait more, save all your files and launch your application. Remember that you can use the command mix phx.server to launch the application, if your application was not yet running.

With this we completed the first part of the walkthough. We will continue with the rest of the views during the lab sessions.