Building A Questionnaire Site With Node.js

Following the same patterns, we wrote down the code for Pug, to render the forms to accept the user inputs. The variable for the total number of questions controls how many questions to render here. Since our form supports 10 questions, we kept a variable holding value of 10 everywhere to store the form and to ask the questions. The variable alteration means that the number of questions on the form would change.

Figure 4: Questionnaire to be filled, with autogenerated questions and options.

Note
All of this code is available in the GitHub repository, as well as on the code sample provided here.

Other minor pages, such as the pages to show how many times a form has been filled, and what are the stats for the page were generated. These pages use the same Pug framework. You can review the pages in the source samples. In the next sections, I will close the front-end portion of the article and write up for the back-end.

Bootstrapping

As seen above, the bootstrap was used in the layout file to be streamed down to every file. Bootstrap is a responsive framework and the only framework of JavaScript that I can infect my web apps with. The bootstrap happened to serve as the primary and the core JavaScript framework for our front-end. We heavily utilized the framework, to support desktop, mobile, and other screen variants.

We used the CDNs services for bootstrap, to load the Bootstrap JavaScript and CSS files. This would end up with removing some load from our server and improving the performance as well. Finally, we did not need to integrate anything, as we were just loading it — they are .min. files, because we do not need to debug their code at all, so why waste KBs?

Bootstrap has some good and useful stuff, that you definitely should look into. Such as the buttons we built, or the forms and the input fields we built. They were developed with bootstrap and the default styling was considered to support responsiveness. To support, we just ran a simple Google test to determine our website is mobile friendly or not, the response was,

Figure 5: Google test for the mobile friendliness of a web page. We used Bootstrap for responsiveness and some extra ingredients to support the purpose.

The problem with the URLs loading is the off-site URLs for other stuff applied on the web page and has no direct relation with the website's responsiveness. This demonstrates that the website is capable of being SEO friendly. Besides, this had the least hard work of ours, because most of the work was done by Bootstrap, so credit to them.

Back-end stuff

That part was the front-end of our app, the backend stuff is a bit complex and complicated. In this section, I will cover up the stuff, such as database management, routing of the application, error handling and a few other things that need to be mentioned. Node.js runtime provide an excellent way of writing the apps, it is performance efficient, and community-supported, which means that it can provide you with a library for almost anything.

The overall setup we ended up with after the app was running looks something like this, of course, this is just an overview of the design,


Figure 6: Overall structure of the app, from connection to backend management.

There are different components that run, in order to make the app live and bring the functionality. We have looked into the front-end techniques, now it would be a good time to look at the backend stuff.

Routing

Before I talk about the database stuff, I would like to share the routing of the URLs and mapping of the HTTP verbs. The URLs of the app are the main part, you cannot expect your app to be used efficiently if your URLs are not well-formed. For the same reason, we tried to use as little URLs are possible and then map the rest to the suitable URLs, such as 404-page. The Express framework provides the services, that you can use to easily map the URLs to your functions, and handle the HTTP requests in them.

We are using the Express framework, which enables us to utilize the HTTP verbs as functions, to map to a URL. To create a server, start with the most minimal approach of a Hello World,

Express Framework supports the HTTP verbs, to attach the functions to each type of URL, based on the request type, such as app.post, app.put etc. You can also use app.all to map to a URL without prejudice for any HTTP verb. In a similar manner, we also configured the routing to support an HTTP GET and HTTP POST for the pages where we had to manage and configure the forms from.

The structure was something like this,

There are a few things to note here, first of all, the parameters in the URL can be specified like, /:paramName. You can then later on access the parameter in the req.params property. This can some real handy if you want to design the URLs to be, /resource/id, instead of having the URLs like /resource?id=value. We followed the same practice and enabled the URL mapping of parameters and optional values, this lets us manage the form IDs and then provide the dynamic content based on that.

Routing can be done in other formats as well, there are controllers in Express framework that you must look into if you want to build enterprise-ready apps, controllers can help you bring the functionality in a layered form.

SQLite database

Node.js runtime supports SQLite 3 databases and provides a library that can be used to work with databases. The npm can be used to install the library.

Either, install via the npm CLI,

Otherwise, install via adding a dependency to that package in the package.json file.

At the time of current writing, the latest version is 3.1, so it would be best if you can check against the npm tools and CLI to determine the latest version of the package for your app. If you do this, even then you would need to update the packages by running the npm update command. The SQLite 3 library contains everything from the database drivers, to connection management to query executors. The complete information for the package can be captured from here. This library is also asynchronous, and does not block the Node.js runtime code during execution of database-bound operations. This means that you can use the JavaScript promise API to chain the SQL operations, and then execute them one after the other.

The topics to cover here in this portion are,

• SQL queries in SQLite3 library
• Parameterizing the queries — again, to best avoid any kind of misuse, consider an ORM wrapper.
• Executing queries further, only if the first query executes well
• Capturing data in different types of boxes (single record, multiple records)
• Managing when no data row is to be returned.

Starting off, I will assume that you have the basic understanding of SQL queries. You start off with the initialization of the database engine, and then you manage the tables.

If you cannot import the sqlite3 package, make sure there is a dependency logged and then update the package manager. First we move on and define out tables,

SQLite supports the queries such as IF NOT EXISTS and IF EXISTS, that can be deemed helpful when you are trying to create a table and want to make sure it only gets created if it does not exist. And same for the IF EXISTS, to delete only when there is something to delete. And during debugging, it might be useful to delete the tables on every restart, for that that removeTables variable was used.

Then, the library also supports different functions that you can execute to select the values. A simple rule of thumb is,

• Execute get(), when you want to get only 1 record from the table, such as a top-level record.
• Execute each(), when you want multiple records but are unsure of how many there'd be. This can help in ignoring a load of data being pushed all of a sudden and the latency to capture multiple records.
• Execute all(), to capture all the records from the database in one go.
• Execute run(), to insert the records and perform another sort of functions, or to create tables etc.

To demonstrate this, consider the following function to capture the solutions for a form,

Like you can see here, we started off with the request and captured the formId from the URL itself. Then, the queries move in and provide us with the form. The signature holds,

The same can be seen above, thus we check if there were any errors in the transaction of the query. If there are, we return the error message. The object returned contains the columns in the form of properties, you can access the properties in a number of ways. To demonstrate this, let me tell you a few of examples,

I used different column labels to demonstrate how JavaScript can let you capture the values of that object in different manners.

The rule is that if a column name doesn't follow JavaScript naming conventions or that is not a legal JavaScript variable name, then you can use the index notation of capturing the variable names, as in the case of last two.

Rest of the SQL queries, and how the form works would take a longer content, where the post is already huge. Thus, if you would like to read how I managed to write the SQL scripts, please read the repository code. Also, the SQL code is not that much important because it is recommended to consider an ORM, which is what we recommend. The repository of the project will also upgrade to using an ORM and layered structure for the forms and data orientation.

The asynchronous approach or writing the code and managing the chaining would take too much of article scope to explain, thus you can try to understand how to write the code, with chaining. The SQLite 3 library that we are using has this asynchronous approach built-in.

Note
It is recommended to consider an ORM, such as Sequelize. Hardcoding the SQL queries is not a bad practice, it only takes more time to do the same. With ORMs, you also have a safer environment, because you do not write any bad code — unless of course the bad code was written in the library, in which case, you're doomed.

Delivery of content as model to Pug

Last but not least, the model passing to the Pug template is the part, where you pass the forms and other information to the templated file. I skipped this part, from the databases section and I am going to talk about how you pass the variables to the template file.

The simplest way is to pass the second parameter to the render function, that would define the properties you can access.

Take the page above, for example, we are passing the solutions and the ID of the form here. To render them, you can access the properties directly — without calling any Model, or other objects first. Such as -

This would create a div element, with the ID of stats, and then iterate and add a child-div element for each of the solutions. A few tips to get you started in this would be,

• If you want to pass an array of elements, pass them as a property of the object, { elements: list }, then access them via elementsvariable in Pug.
• For other fields and variables, pass them as a property directly and access via the name.

There is no need to wrap or write any special model imports in Pug template, the overall object that you pass as a parameter gets exposed as a model and you call the properties and render them. You can also see the same pattern in the code above, solutions[i].by, where solutions were the list, and the by was a field to render who filled the form with this solution. To learn more about this, consider learning more about the res.render() function of the Express framework and how you can pass the model parameters.

Deployment

Now that our app is up and running, it is time to host it. As mentioned, we wanted to try something else, something other than just-Microsoft. So, we worked out an account on Heroku platform and setup the verifications/other formalities to have our app run smoothly.

Heroku somehow let us down, provided the speed and performance of Node.js framework. Understood that I am using a free account, but even then it must not be as much bad as it sounds — after all, I would have just used Azure for this and I would have at least x10 times better results than this. Nonetheless, the deployment models of Heroku platform support an overall three models,

• GitHub
• Dropbox — Heroku CLI falls back to this deployment model.
• Windows Containers

The following image demonstrates this,

Figure 7: Heroku dashboard for deployment of apps.

What I consider a good approach here is the Heroku CLI approach. There are several benefits of using Heroku CLI instead of GitHub or Dropbox. Heroku CLI uses the Dropbox folders that you authenticate against, then it uses the git version control to manage the repository and deployments. I used GitHub, the repository for it was public, and can be accessed at https://github.com/afzaal-ahmad-zeeshan/friend-knower, there were some common problems with this, such as all your code is public, unless you have a premium GitHub account. Which I cannot afford for this. I had some code integrations for other purposes, such as ads, which I cannot afford to share on GitHub — even though it is not a big deal for your ad units to be left in wrong hands, as Google will not bother delivering ads to those websites.

Heroku CLI is a very simple, intuitive and straight-forward command-line interface for managing your applications. The overall workflow to,

• Authenticate your account
• Connect to the Heroku repository for your project
• Deploy and publish the app

Would be just the way as follows, to authenticate, you run the following command,

This will ask for your Email and Password to authenticate against. Once that has been done you just need to clone your repository and commit the changes to the same repository, which will publish your application.

Heroku CLI would generate the heroku remote link to git repository, which is why you can push to this repository. It is similar to your own repository on Heroku, either connecting directly or through this method.

A live demo of the app can be checked out at, https://friend-knower.herokuapp.com/.

What we learnt

Among many things that we discovered the primary thing was: Abandon JavaScript!

JavaScript is a dynamically typed language, thus it gives you an opportunity to write bad code. If you have programmed in JavaScript, then you know that you are typically prone to errors like, undefined type or not found and much more. Not just this, there is no type safety in JavaScript. You can manually add conditions like, instanceof and then continue writing the code which would be type safe, but other than that, there is no type safety thus you write code with more if...else blocks to check if everything is Okay.

In this case, we realized the fact that TypeScript can help us in writing better code for Node.js. We will try to write another article about something in in Node.js but with TypeScript in future.

Figure 8: TypeScript logo.

Another major thing we had learnt was, that you should consider using an ORM in Node.js for the database platforms that you are trying to target. We discovered Sequelize framework, which is quite good and provides an excellent support for generating the models and managing the overall DDL and DML experience. Consider visiting their official documentations for learning how to write the ORMs for Node.js. Also, Sequelize supports multiple database dialects,

• SQLite
• PostgreSQL
• MySQL
• SQL Server

An ORM can help you out with a lot of stuff. We somehow managed to have the app built with minimal coding and thus integrating the ORM would've not made any difference for our sample. If you are trying to write a large app, we recommend that you use an ORM — you are free to make any choice, Sequelize is just what we felt good with.

For the platform of Heroku, we learnt that they do provide a well formed service. They provide excellent deployment models, with multiple ways to deploy the app. Since they support GitHub, they support automatic integration (continuous integration happens on GitHub). This was a very easy model for deployment of the apps. See the Deployment section above to learn how the process works.

What's missing

This application was merely a sample run down for the overall concept of questionnaire app. Though we tried to invest a huge amount of time in building a good starter template, but we were unable to determine whether this concept would be adopted or whether it would be a waste of time. Thus, we did not add more improvements to it, if you feel like this has to be upgraded, let us know and we can start the platform.

Our focus in the upcoming versions would be to generate templates so that forms are autofilled by others, and you only pass the name to write on the form, we also want to integrate the SQLite ORM library in this, that will help us in removing the problems such as writing/changing the queries each time our model changes.

Lastly, we want to write handlers for states where a form doesn't exist, a question doesn't exist and much more. Our current version doesn't support that.

Lastly, we also to support multiple question types, such as checkbox-type and text-box type. More over, the number of questions is a hurdle in making this app useful. We want to work this out, and provide an extendable form that can grow/shrink as needed by the form creator. We could really use some of your help in shaping this library/app to extend and provide more functionality. You can share your insights or some help on the GitHub repository.