Static Signal

Express is awesome.

I’ve worked with a few backend technologies, but Node will probably always be my favorite because JS was the first language I really picked up.  Python (Django) and C# (.NET) are both cool, but being able to write JS on the server is great- no more context switching and ending my Python lines with semicolons!  You can create some really cool stuff with Node if you mind the event loop.  If you’re experienced coding for a browser, it shouldn’t be too difficult- the principle of not blocking the loop applies in both places!

Anyway, back to Express.  It is a very useful wrapper for Node- making it easy to handle requests and send responses, set headers, and even bring in security packages (hello helmet!).

One of the best parts is the static file serving process.  As a learning exercise, I wrote a Node process to serve static files- it works, but it’s not pretty, and I’m sure it’s not as robust as Express offers.  Just do the normal Express app setup, then call:

app.use(express.static(path.join(__dirname, 'directory-path-here'));

If you want your code to be a little more flexible, use Node’s __dirname global.  It gives you the directory name of the current module- very useful to make sure your code is portable (for running in dev vs production, for example).

However, there is one little “gotcha” with serving static via express- particularly when creating a single page application.  express.static can be passed an options object as its second argument.  Without anything passed, it uses sensible defaults- one of which is looking for an ‘index’ file in the directory it’s given.  Makes sense- it’s a good place to start when looking for what file to serve.  If you’re creating a single page application, you probably want to return index.html for just about everything.

But we wanted to include a nice alternate homepage for people using very old browsers (think IE9 and below).  Those won’t load our fancy Angular application, but we didn’t want them to just see a page of gibberish.  So we created a simple HTML page with some vanilla JS interactions on a contact form (felt like the good old days!).  But when we went to configure the routing, we just couldn’t get the server to return anything but index.html- even when we set a conditional in our route to check the browser name/version and return a different file.

The answer was in the documentation (of course).  That options object can be passed an “index” property.  This is a boolean that defaults to true, but if you set it to false, Express won’t automatically serve up the index.  You can control what gets served- just remember that now you have to return index.html manually (after any other possible files, depending on your setup).

app.use(express.static(path.join(__dirname, '/'), {index: false}));

There we go!  Now people forced to use IE8 (and I can’t think of anyone who would voluntarily use it these days…) can still view our super important content!

Living in a Material World

The more I work with React, the cooler it seems.  I know- I’m way behind.  React is already uncool, Vue is the new hotness.

However, every time I start writing custom HTML components in plain ‘ol Javascript, I end up with a pattern that looks a lot like React.  I then say to myself: “Why am I recreating a crappy version of React instead of just using React?  People smarter than me have spent many hours making a much more optimized version of this monstrosity I’m building”.  Then I look around to make sure no one heard me talking to myself.

It’s at that point that I run the “create-react-app my-app” command and really get down to business.

I started a project with React almost a year and a half ago, but just couldn’t wrap my head around the router.  It’s not that React Router (v4) wasn’t well built (though I might have thrown some insults at it in a frustration-fueled rage), but I’d been using Angular at work, and the router philosophies are pretty different between those two.

I started a new project with React (integrating some Phillips Hue lights into a ui) a couple weeks ago, and decided to give it all another shot.  The first few hours of figuring out the routing was still a struggle, but then it kind of clicked.  The <Router> declaration in React is a bit like the router-outlet in Angular, with the declaration of the route all in one.  I don’t think I’d really grasped that originally, and it led to a mess of an application.

But the router isn’t the point of this short post.  This one is about integrating Material-UI with React and how cool composable components are when you start to understand how to really use them (note that I said “start to understand”- I have a long way to go).

So, I’d already created some simple components to get everything working.  I’d also cooked up a simple Node server to serve as a fake API.  I hadn’t bought the smart lights yet- but wanted to test, so I just copied the JSON structure from the documentation, pasted it into a file on my computer, and set up a server on localhost to respond to the same endpoints the actual API would and serve up the JSON.  It works surprisingly well!

One of those components displays all the lights available (2 currently).  That LightPanel component is made up of LightSwitch components- one for each light.  The focus early on was to get them working.  Once I reached that point, the time had come to actually make them look presentable.

Too often this is where I end up burning a lot of time.  I have a habit of trying to create nice looking css on my own- but this time I learned my lesson.  I’d let the experts help out, and use the Material UI library.  It integrated pretty easily, As a test, I switched my LightSwitch jsx from simple custom radio buttons to the Toggle component and it instantly looked better.  Success!

But the really cool part was when I decided to use the Card element for my main LightPanel grid.  Again- this will show all the switches, their status, and allow a user to turn them on or off.  Each light gets its own Card in the grid.  The functionality to get this info and do these actions was already there, I just had to integrate it into the Card element.  I thought the best way would be to just embed my LightSwitch component into the Card’s ‘text’ section.  This worked, but didn’t look quite right.  I realized that it would really go best in the Card’s ‘title’ section.

But the title section is meant to be passed info as attributes- not have text inside tags.  Instead of <CardTitle>My text here</CardTitle>, it should be <CardTitle title=”My text here” />.  I wanted to basically embed a custom React component into the “title” attribute.

Thinking “there’s no way this will work”, that’s exactly what I did:

I saved and waited for my command line output to give me a wall of red text, but none appeared.  Fingers crossed, I opened my browser, and there it was, in all its glory.  The card with my custom switch component embedded as the title attribute- toggle functionality and all!

So thanks all around: to React, to Material UI, to the wonders of Javascript!

When in Rome

I really like learning new programming concepts and methods.  The beautiful and frustrating thing about software development is that it’s kind of endless.  No matter how much you learn, there’s almost always going to be something you don’t know.  As long as you don’t let that though overwhelm you, and focus on the process instead of the outcome, it can be a fun ride.

And the internet is a great place for learning.  Don’t get me wrong- it’s also full of inaccurate, opinion-tainted drivel, but you can definitely find really good resources.  How to tell them apart?  The compiler (or browser in the case of the web) doesn’t lie.  Learn something, try it on your own- if it works, on a real project you’ve found a good resource.

A couple months back I posted about Wes Bos’ great courses (seriously- if you want to learn, go get his stuff).  I also find Pluralsight very helpful (sometimes you feel like laying down and being lazy- why not watch videos too!).  Free Code Camp is another great avenue for learning.  I started that track a few years ago but drifted away due to work and time limitations.  When I went back a few weeks ago, I found even more great problems and projects to practice on.

One of them is the Roman Numeral Conversion algorithm problem.  It’s listed as an “intermediate” difficulty question, but the instructions are simple: “Convert the given number into a roman numeral”.  For some reason, this one was giving me a really hard time- I just couldn’t figure out how to do it without making a big map of numbers linked to their corresponding roman numerals.

So I decided to do exactly that.  I would solve the problem in the worst way I possibly could.  If my only idea was to make a big map, I’d just make a big map.  And something interesting happened along the way.  Patterns began to emerge.  Using my big map, I had a bunch of conditional logic, depending on the length of the number.  But I realized I could break that into a single loop.  I had a whole bunch of logic in that loop that really belonged inside a helper function, so that was the next step- abstract it out to a helper.  Finally, I realized that instead of the map (much smaller now), if I used an array, I could match the index up with the loop counter variable in my main function- passing the correct Roman Numeral letters without having to manually do so each time.

The code is available in some Github Gists- first, second, and final passes.  Apologies if there are errors- never used the Gist feature before, but it’s pretty cool!

I bet there are many ways to improve it- the logic in the helper seems unwieldy, but the best thing I learned from this exercise was the general process.  Get something working, see patterns, edit, repeat.

Strippin’ Tags

A while back, I helped create a Django back end for an update of an existing website.  The website’s front end was AngularJs, and we didn’t really get how the two should integrate.  The result?  A bit of a mess.  Django template views inside Angular html template files.  We had to modify the bracket style used for Angular (as it conflicts with Django’s).  The folder structure was confusing, the code was confusing- it worked, but it wasn’t a great solution.

Now I understand what we should have done: use Django to create an API for the Angular front end.  Just return that sweet, sweet JSON to the app and have Angular do the templating.  We are in the process of this update (as well as migrating this one from AngularJS to Angular the Next Generation, and I ran into an interesting project.

The original site had a Django-driven blog.  This was not integrated into the main AngularJS app in any way- a user clicked the ‘news’ link and they were taken to a completely new page with a traditional Django blog setup.  It made creation and updating easy- we just used Django’s admin panel for new posts, and the default templating views to handle any sorting (by category, date, etc).

But it doesn’t really fit.  Having the blog as a module within the Angular (now to be v4.2.4) application makes more sense.  With our new API approach it will work, but it will take some extra work.

One aspect is the admin panel.  We won’t be using the built in Django admin panel- instead, we’ve created an Angular-driven admin panel.  Converting the data returned by Django’s ORM to JSON was a bit tricky at first, but it’s flowing smoothly at this point (might cover that in a future post, as it was a bit of a process).

Another aspect is searching through blog posts (on the user interface), and that’s where we come to the cool project of the week.  One of the benefits of doing this extra work is getting the hip “instant updates” feel of a single page app within our blog’s UI.  When someone types in the search bar, they see the blog list below filter immediately.

But I noticed some test posts were coming up in almost all search results.  They happened to be the test posts with images in them.  The reason?  We are using an HTML editor toolbar for the admin area.  When a site admin posts a new blog, they use this toolbar to format text, add links, or post images (not everyone posting will be a developer or have source code access).

The toolbar has a cool feature where it encodes any images uploaded to base-64 format.  According to Wikipedia,  “Base64 is a group of similar binary-to-text encoding schemes that represent binary data in an ASCII string format by translating it into a radix-64 representation.

I don’t really know what that means, but I do know it means I don’t have to set up an ‘images’ folder for my blog module and save a path to those images with a relation to the blog post in my database.  When it converts to base-64, it gives me a string of text that a modern browser can translate into an image.  That image src can be saved right along with the rest of the HTML body.

Very helpful – but back to the search feature.  It turns out that the search was iterating over everything stored in the blog’s body- including the html code.  I would have had to fix this anyway, but it became really apparent with the image issue.  Because the string of text representing an image was so long, it naturally contained matches to most strings I was searching (at least the simple ones people would start with).  It might not have mattered on a traditional request-response site- a person would type their whole search string then submit and avoid seeing the wrong results.  But in a live reload search, the problem was obvious.

My first thought was a terrible one: Why not store a plain text version of the blog body in the database?  That way, we just return it alongside the rest of the JSON data and use that for the search.  But that really doesn’t make sense- it’s just bloating our database tables with duplicate info and increasing the size of the JSON object returned on each request.

Then I remembered how cool Javascript can be- and that it comes with awesome built in array features like ‘map’.  So, why not return the data as we have been (with the body as HTML) and manipulate the body just within the search function?  We can search over that manipulated body, but display the original HTML.

Turns out that this works pretty well.  In our main blog component, we initialize a search array on a service:

this.blogViewService.searchArray = ['author', 'title', 'body_plain_text', 'category'];

This is a convention we’re using on a different project- the array contains the property names we want to search by (those property names appear within each object in the array we will be searching over). That searchArray is passed to our search service with some other info.  In this case, ‘body_plain_text’ doesn’t exist on the object- but we’re going to create it as we go.

For the search, we cheat a bit and use Angular’s built in form input subscription.  A form input field in Angular has an observable you can hook into to get the data as a user types (called valueChanges).  You can then subscribe and do any searching there.  All we need to do is make sure to transform each object a bit in the process:

this.blogViewService.searchSubscription = this.blogViewService.term.valueChanges
    .subscribe(result => {
        let itemsToSearch = this.blogViewService.originalItems
            .map(item => {
                item['body_plain_text'] = String(item['body']).replace(/<[^>]+>/gm, ' ');
                return item;
        //we pass to another service to do the actual filtering of originalItems here

We start with assigning a reference to this subscription (searchSubscription) so we can unsubscribe on the component’s destruction (to avoid memory leaks).  Then we hook into the form input observable (valueChanges).  I put a slight delay on the process at this point with debounceTime(200)- when the news/blog section gets big enough that returning it all up front doesn’t make sense, we will have to hit the database in this search.  debounceTime(timeHereInMs) is a great rxJS built in that handles debouncing your calls.

Finally, we get to the actual change- the originalItems array is mapped, but none of the properties within each object are actually changed.  Instead, we append a ‘body_plain_text’ property to each object that uses a regex to strip HTML tags.  Originally, it replaced with nothing, but then we had words joining together (if they were on the other side of tags), so replacing with a single blank space preserves the integrity of the search.  We never change the original ‘body’ property- this is where our HTML lives and is used for the actual display.

I’m sure there will be edge cases where this might not work and we have to tweak the process, but it’s a good start.  I also don’t think this is technically how you’re supposed to use .map- it’s a functional programming staple, and I’m using it to append a property to an object then return it back to the array.  Definitely not functional programming!

Jumping in the Deep End

I was (kind of) wrong in an earlier post.


That pause was to allow everyone to recover from such a shocking confession.  Everyone ok?  Ok- now we can continue.

Earlier this year, I wrote a post about making sure you’ve made a proper copy of an array or object in Javascript if you need to keep the original around.  Javascript objects and arrays (and arrays are really just objects in JS anyway) are pass by reference- so just setting “let array2 = array1” won’t work- any edits to array2 will also affect array1 as they both reference the same array/object.

So, you create a ‘deep copy’ of the array- the easiest way in ES6 being using the spread operator.  Just call “let array2 = […array1]” and you’re good to go.  Working in an older environment?  Use “var array2 = array1.slice(0)”.  Up to that point, my earlier post was correct, but it didn’t dig deep enough.  My solution to shallow copies was too shallow.

In many web development situations, someone working on the front end is getting data from a database of some sort.  In a current project, this is coming from Django.  The data is JSON serialized, so we parse it and get an array of objects.  Using the deep copy method on the array, we can manipulate the data for display.  As an example: we get some blog entries- then we want to get a listing of year-month pairs with the number of posts that fall into those groupings.  I know we could get this info with another DB call, but it’s already there in our original http request, so some manipulation should work fine.  We take the ‘posted’ value, make sure we just have the year and month, and do a loop to count up the total instance of each.

But that data manipulation occurs on objects in an array.  I created a copy of the array, but each element in that copy still references the original object from the original array.  I call it reference copy hell- I thought I had a fresh copy to work on, and technically I did- the new array was not the same reference, but that fresh copy was packed full of references to the original objects.  When I edited the posted date for grouping and counting, I also altered the original and the date display on the page was altered.

To fix, I tried the functional programming route.  One of the beautiful aspects of JS is that you can pass around functions just like any other parameter.  Arrays even have built in functions for this, like map and forEach.  So, to ensure I didn’t alter the original array or any object within that array, I made:

function copyArrayOfObjects(objArr) {
  return [...objArr]
    .map(obj => {
      return Object.assign({}, obj);

And it’s chock full of ES6 goodness.  This will be for an Angular project, so we’re running through a transpiler and that should take care of any compatibility issues.  This function just takes the original array, makes a copy of that, and maps objects within to brand new objects using Object.assign.  It probably needs some sanity checks- just to make sure it’s being passed an array of only objects.  That would be pretty simple, but not necessary just now.  The only time it will be used is getting an array of objects from the DB, but if it seems useful to extend later, I will definitely revisit.

As an aside- I’m not sure I found the best way to accomplish my original goal.  I wanted to take the original array of objects and extract:
1) an array of category titles and a count of posts that fall in each
2) an array of year-month pairs and the count of posts that fall in each.

I feel like there should be a way to use .reduce to do this, but never could figure it out (still trying!).  What I did was to create another helper function:

function aggregateArrayOfObjects(objArr, field) {
  let aggregated = [];
  let intermediateMap = {};
  objArr.forEach(item => {
    if(intermediateMap.hasOwnProperty(item[field])) {
      intermediateMap[item[field]] += 1;
    } else {
      intermediateMap[item[field]] = 1;
  for(let i in intermediateMap) {
    aggregated.push({name: i, total: intermediateMap[i]});
  return aggregated;

It takes my array of objects and the field to aggregate.  First it generates an object that tallies up the count for each distinct instance of the ‘field’ string you pass in.  Then it returns an array with ‘name’ (for the title of the field instance) and ‘count’ (the total times it was found).  It feels inefficient to loop an array to make an object just to push all that back into an array, but it does work (and I’m on a bit of a schedule with this) so it will have to do for now.  With this function and my original copy helper (yay functional programming!), I can get what I want:

const archiveFirst = copyArrayOfObjects(test)
  .map(b => {
    b.posted = b.posted.slice(0, 7);
    return b;
const archive = aggregateArrayOfObjects(archiveFirst, 'posted');

For the archive (year-month pairs) we just deep copy the original array of objects (in a variable “test” here), manipulate the posted field (it includes the day of the post, but this isn’t really useful in grouping as it will be different for every post), and pass that into the aggregate function with the title of the field to aggregate by.  I know this wouldn’t work for an array of objects if one of those objects also references an object (or array), but I also know my data doesn’t currently do that.  There is probably a solution using recursion to walk through the array copy, look for objects, then look for objects/arrays within that object, and convert to a deep copy, but at that point I’ll just pull in Lodash!

Want to play with any of these?  This link to (an awesome playground for testing code in just about any language) should work.

Taking the Slow Road

The JS30 course from master instructor Wes Bos is really cool.  It includes 30 small* projects that really only involve JS/HTML/CSS- no libraries, frameworks, etc.  I’ve been working as a developer for long enough that I know some of the info already, but even if I see the title of a video and say “Awesome- I know that one!”, I can still learn something useful.  That’s part of the beauty of coding- once you get past the basics of syntax and making things run, there are many different ways one program can be written (and with JS, even the syntax can be a point of departure: semicolons anyone?).  It’s always good to see how someone else does it- you’ll almost always learn something new.

Generally, I’ll check out the first part of the video to see what we’re building, stop it, and go try to implement myself.  Then, I’ll watch the rest after either: A) I finish my version, or B) I get to a point where I’m not sure where to go next.

Example: one of the videos covers creating a cool sliding panel full screen image viewer.  My original version worked (using simple colors instead of images as a background), but I did not know you could animate the flex property on an element.  That’s how Wes did it in the rest of the video, so when I went back to re-do the project, I tried that instead.  It made the code much cleaner- in my original version, the css width values for the panel, grow, and shrink classes would have to be updated if I wanted to have 4 panels instead of 3- and the transition never quite had the exact width property (if you switch hover fast enough, you see the background image).  Also, I know animating width is generally not a great idea for performance as it triggers the unholy trinity of layout, paint, and compositing in the browser.  A quick check of tells me flex does the same, but I tried using a more performance-friendly transform (scale) and it just did not look right, so a little less than 60fps will have to do!

If there’s one takeaway from this post, it should be that there are many paths to a working program.  I definitely don’t always find the best way on the first try- but if I approach the problem with the mindset of getting it working, then iterating until it works really well, I usually get the best results.

If there’s another takeaway, it’s that Wes’ courses are awesome (I learned most of what I know about React from his great course on that topic).  Some are free, some are paid- all are well worth doing.

But the final takeaway (I know, that’s three now) involves that asterisk from the very beginning of this post.  The idea behind JS30 is to do one project per day for 30 days.  And if you stick to just the parameters of what Wes lays out, you can definitely do that.  But it’s so easy to get carried away.  Is a program ever really done?  Eventually, you get to a point where it works and works well and you ship it- but it seems like there are always improvements that can be made or new techniques to be learned.  At work, you usually have someone reminding you that there is a deadline for a project, but with side projects, it’s easy to give in to scope creep.

Sometimes it’s not too bad.  For my updated version of the sliding panels gallery project above, I did a little mobile version that simply switches the flex direction to column and transforms to horizontal.  I also added a slight ‘screen’ effect over each image to allow for the text to show a bit better on the images I chose (learned this can be accomplished using a linear gradient as the background image).

//creates a soft darkening on the bg image without affecting the text color as opacity would
background-image: linear-gradient(rgba(0,0,0,0.3),rgba(0,0,0,0.3)), url('./img/tree.jpg');

On that same project, I decided I wanted a user to be able to click/tap any panel to have it open and have all others close (even if one is already open)- more of an accordion effect.  That was just a matter of adding a simple JS function:

function removeClassFromArray(elementArray, className) {
    elementArray.forEach(function(ele) {

I added it to the click handler for each, before the line where I set the ‘open-active’ class to the panel that should be in focus. This ensures only one panel is open at a time.

So, nothing too crazy- still a one day project.  But then there was the Fetch API project.  I feel like I know Promises and working with async code pretty well, so initially this one was going quickly.  However, as I finished up, I realized there was so much left to do!  A user can get the city population, sure- but what then?  Well, adding the total result count and total population wasn’t too hard.  But the total population number looked off.  There were no commas- so I created a function to add them (likely point for future optimization):

function numWithCommas(num) {
    let strArray = String(num).split('');
    let withCommas = [];

    var count = 0;
    for(let i = strArray.length - 1; i >= 0; i -= 1) {
        if(count === 3) {
            count = 0;
        count += 1;
    return withCommas.join('');

Cool, so now we have a running total of results and the total population from all the results.  That’s not super useful though.  Wikipedia has a pretty cool API, why not grab info from that if they click on any result.  That will provide a blurb and link to the full article.  And we should have some kind of notification or feedback when info loads or something goes wrong. And stick the input to the header if they scroll down a huge list of results.  And if we put a listener on the scroll event, we will need a debounce function to not harm performance (that one’s very interesting- might dig into it next week).

You see where this is headed.  The todo list on the Fetch project is still growing to this day (mobile, better animations, etc)- though I’ve moved on to other projects for now.  JS30 has become more like JS300, but I feel like I’m still abiding by the spirit of the exercise.  That is, daily exercise for your coding muscles.

Light My Firebase

Last month, I finally carved out some time to build my own personal website.  Way overdue- but anything I create instantly looks like a pile of crap, so I usually end up spending way too much time trying to tinker with the look.  What I really love is making things work.

I used Angular to continue practicing that framework- particularly with the new animations module.  It’s pretty fun and easy to use.

I decided to host on Google’s Firebase platform.  I always thought it was just a real-time database, but now they offer content hosting as well.  With their tools you can install locally, it makes deploying super easy (as in: type “firebase deploy” level easy).

But what about a backend?  I wanted a simple contact form, so I’d need a backend to process the email sending (submissions would also be stored in the real time database).  I love writing Node- but most of my experience so far has been with Express- I wanted to get into the basics.  http, fs, dns, etc (that last one isn’t a module, don’t try to require it into your project)- all very useful and all native to Node.  My first thought was to just make a simple node backend with just a couple endpoints: one for sending email (using the awesome nodemailer library) and one for returning all routes to the root (to serve my Angular project).

However, Firebase hosting doesn’t work like that.  It’s not so much a backend hosting as a serverless architecture.  Please note: I may be using that term incorrectly.  In this case, it just means that you don’t use http.createServer and listen for request events.  Nor do you import express and build an awesome rest API.  With Firebase hosting, you use functions.

And those functions really are basically just simple Javascript functions.  In your app’s main folder, create a functions folder.  You can have multiple files, but be sure to have an index.js file.  This is where you can require other modules- or if you’re just making a simple function (like mine to send email), insert your backend code there.

You will also have a package.json in your functions folder (I believe Firebase creates this for you when you initialize their Functions option)  It will have a few dependencies by default (firebase-admin so you can authenticate your app and firebase-functions so you can register your functions).  You can add other Node modules here as well- I added nodemailer.  These become available as ‘require’ able in your index.js.

I created my email sending function- most of it is configuring the sending options and the template for the email content, as well as some nodemailer related config.  The content isn’t important- it’s just titled sendFormHandler.  Once it’s done, I just register it with an event on my database (in this case, saving a new contact on form submission):

exports.sendFormViaEmail = functions.database.ref('/form_submissions/{pushId}').onWrite(event => {
    const newSubmission =;
    return sendFormHandler(,, newSubmission.content);

exports is just a shortcut for module.exports. functions is a reference to require(‘functions’).  I grab a reference to the data (the event is the database write, the data.val() function gets the object submitted.  Then I can pass that info to my simple sendFormHandler and off we go!

I’ll still stick to writing backend code for more complex applications, but this serverless thing is pretty cool (if that’s what this qualifies as).  One mistake I made you might avoid with Firebase: double check the path you provide to functions.database.ref.  Initially I was passing just ‘/form_submissions’ – the endpoint I use for saving new submissions.  However, this gets all entries in the form_submissions object (I almost said ‘table’, but this is not sql!).  I spent more time than I want to admit logging the return value from that endpoint, trying to figure out how I could get a diff of the previous object and the updated object to single out the new one.  I found a property: that showed the new object, but the pushId is randomly generated, so I couldn’t figure a way to reliably select just that object.  However, adding ‘/{pushId}’ to the end of the url you pass takes care of all that for you.  It gets the most recent entry by that unique pushId- very useful! only new addition, but had trouble accessing properties (pushId is randomly generated).