Light My Firebase

Last month, I finally carved out some time to build my own personal website.  Way overdue- but anything I create instantly looks like a pile of crap, so I usually end up spending way too much time trying to tinker with the look.  What I really love is making things work.

I used Angular to continue practicing that framework- particularly with the new animations module.  It’s pretty fun and easy to use.

I decided to host on Google’s Firebase platform.  I always thought it was just a real-time database, but now they offer content hosting as well.  With their tools you can install locally, it makes deploying super easy (as in: type “firebase deploy” level easy).

But what about a backend?  I wanted a simple contact form, so I’d need a backend to process the email sending (submissions would also be stored in the real time database).  I love writing Node- but most of my experience so far has been with Express- I wanted to get into the basics.  http, fs, dns, etc (that last one isn’t a module, don’t try to require it into your project)- all very useful and all native to Node.  My first thought was to just make a simple node backend with just a couple endpoints: one for sending email (using the awesome nodemailer library) and one for returning all routes to the root (to serve my Angular project).

However, Firebase hosting doesn’t work like that.  It’s not so much a backend hosting as a serverless architecture.  Please note: I may be using that term incorrectly.  In this case, it just means that you don’t use http.createServer and listen for request events.  Nor do you import express and build an awesome rest API.  With Firebase hosting, you use functions.

And those functions really are basically just simple Javascript functions.  In your app’s main folder, create a functions folder.  You can have multiple files, but be sure to have an index.js file.  This is where you can require other modules- or if you’re just making a simple function (like mine to send email), insert your backend code there.

You will also have a package.json in your functions folder (I believe Firebase creates this for you when you initialize their Functions option)  It will have a few dependencies by default (firebase-admin so you can authenticate your app and firebase-functions so you can register your functions).  You can add other Node modules here as well- I added nodemailer.  These become available as ‘require’ able in your index.js.

I created my email sending function- most of it is configuring the sending options and the template for the email content, as well as some nodemailer related config.  The content isn’t important- it’s just titled sendFormHandler.  Once it’s done, I just register it with an event on my database (in this case, saving a new contact on form submission):

exports.sendFormViaEmail = functions.database.ref('/form_submissions/{pushId}').onWrite(event => {
    const newSubmission = event.data.val();
    
    return sendFormHandler(newSubmission.name, newSubmission.email, newSubmission.content);
});

exports is just a shortcut for module.exports. functions is a reference to require(‘functions’).  I grab a reference to the data (the event is the database write, the data.val() function gets the object submitted.  Then I can pass that info to my simple sendFormHandler and off we go!

I’ll still stick to writing backend code for more complex applications, but this serverless thing is pretty cool (if that’s what this qualifies as).  One mistake I made you might avoid with Firebase: double check the path you provide to functions.database.ref.  Initially I was passing just ‘/form_submissions’ – the endpoint I use for saving new submissions.  However, this gets all entries in the form_submissions object (I almost said ‘table’, but this is not sql!).  I spent more time than I want to admit logging the return value from that endpoint, trying to figure out how I could get a diff of the previous object and the updated object to single out the new one.  I found a property: event.data._delta that showed the new object, but the pushId is randomly generated, so I couldn’t figure a way to reliably select just that object.  However, adding ‘/{pushId}’ to the end of the url you pass takes care of all that for you.  It gets the most recent entry by that unique pushId- very useful! only new addition, but had trouble accessing properties (pushId is randomly generated).

SPA Day

Picture yourself at a spa.  You’re relaxing in a nice hot tub, letting the worries of the week soak away into the scalding water.  Your phone rings – a flare of stress and responsibility is fired off into the night.  Do you really want to get out and answer that call?

Now, I’ve never been to a spa, but I have been in a couple hot tubs in my day and the idea of popping out into the cold just to answer a call is not appealing.  But, metaphorically speaking, this is exactly what I’d made our Angular application do.  It never complained, but I still feel bad- it wasn’t very thoughtful of me!

Let me explain- the spa in this case is an acronym – Single Page Application.  You may be familiar with this term- it’s an app that basically lives on the client/browser.  The bulk of the app’s structure and logic/functionality is downloaded up front as a bundle of Javascript, then network calls (think AJAX) are used to get data as needed.  A different model from the traditional request-response website, where each page is fully fetched, data and all, from the server.  You trade off a possibly slower initial load time (though that’s becoming less of an issue with tools like the Angular CLI) for a lightning fast app after that first load because you rarely have to go back to the server for more info- it’s all there in your bundle.

So, the metaphor does kind of work.  I’m the application, the hot tub is your browser, and once it’s loaded, why make it leave?  Of course, sometimes it has to make an important call.  If anything is going to be saved to a database or read from a database you’ll have to make that network call.  But the more you can minimize these, the faster your app will be.

On many views of this SPA, we show a grid of data.  That’s a trip to the database- no way out of that one.  There’s also an “edit” link on each row of the grid so a user can update the details contained within.  Initially, we had these going to the database to get the details of that row.  But that’s the old way of thinking!  That trip isn’t necessary to get anything to be displayed on the screen- it’s just to get data that’s technically already available (in the array of objects that’s fetched to populate the main grid).  All we have to do is pass it along to the individual view.

For our Angular application, I initially thought this might be an option on the Router class.  You can pass params to a child route, but that doesn’t really work in this case.  The data needed by the edit screen (child route) can be quite a few properties long, and passing all that in the query string didn’t seem like a great idea.  But this (sharing data between two components) seems like exactly the kind of thing services were made to do!

In the grid component’s template, there is an “edit” button.  When clicked, it fires a method on the component’s class to navigate to the edit component route and originally just passed the id/key of that item.  Now it’s been updated to pass the key and the object itself- the info from that row in the grid.  This object is passed to a service that has some other utility functions on it, but the one we need in this case is super simple:

setCurrentEditObj(obj) {
    this.currentEditObj = obj;
}

The currentEditObj is a class variable that’s just used to hold the object we’re about to edit.  This simple method updates that object any time the “edit” button is clicked on a grid.

In the edit component’s class, we subscribe to the route’s params observable.  This fires off info when the route resolves and gives back any params passed on the route.  In this case, it’s usually just the id/key of whatever is being edited.  In the original version, that id is then used in a getItem method- which just does an http call to the database to get the relevant info.  We inserted a check in that observable:

ngOnInit() {
    this.subscription = this._route.params.subscribe((params: Params) => {
        const currentObj = this.setupService.currentEditObj;
        if(currentObj) {
            this.countryName = currentObj['country_name'];
            this.id = currentObj['country_key'];
            this.setFormFieldDefaults(currentObj);
        } else {
            this.id = params['id'];
            this.getItem(this.id);
        }
    });
}

We wait for the route to resolve and check if the service has a currentEditObj available.  If so, we set the class variables (countryName and id for this view, which just edits a list of countries), and populate the form (setFormFieldDefaults just loops through the object, checks to be sure the object’s property exists on the form, and sets the initial value).  No database trip- our app gets to stay in the warm hot tub that is your browser!

But there’s one check you should do.  You might have noticed the “else” branch in the code above.  The internet isn’t perfect- your connection might be dropped and a full page refresh might be required.  A user might accidentally refresh the screen as well.  If that happens, it can cause issues for a single page application, because the object we are editing is now stored in the code on your browser.  If that browser refreshes, that info is lost.

As an aside – this kind of relates to the notion of offline availability.  Some really cool new techniques are being explored in this area using service workers– these can sit on your browser and respond to requests as if they were a server when a server isn’t available, but will be much faster (as they don’t actually have to travel anywhere) and be available if you lose your connection.  Very cool stuff, but not widely supported at this point.

Instead, we just include a simple check.  If that currentObj doesn’t exist, fall back to the old method and use the id from the url (which will still be available, as it’s stored in the url string) to go to the database and get the necessary info (getItem just does that, then calls the setFormFieldDefaults method).  The only time this should be called is if there’s a refresh or dropped connection, but it works nicely- the data is still available, and the extra cost of a database call isn’t really felt, as the entire application needs to be re-fetched anyway.

Does it save a lot of time?  Probably not- none of the objects we are fetching are huge.  But it is a saved network call, which definitely conforms to the spirit of a single page application.

Should I have thought of this originally?  Probably- it really makes more sense within the spirit of a single page application.  Anywhere you make an HTTP call should be examined and evaluated.  Does this call need to be made?  Do I already have access to this info?  Am I making my app leave the relaxing waters of your browser to make that call?  Let the app relax!

Have You Ever Danced with the Format in the Pale Moonlight?

Last time on The DevBlog…

We were formatting some data for display, but it also has to be searchable and sortable by the expected parameters.  Everything was working, but now an additional task appears- add commas to large numbers.

An example is a good place to start.  The column in the grid is for Current Shares.  A person might have 1,021,400.00 shares of something (Beanie Babies, maybe?  Doesn’t matter at this point).  The number isn’t stored in the database as a string- that would make my formatting job easier, but would make calculations on shares a bit unpredictable to say the least.  So, we will get something like 1021400 from the database and format it for display.

Perfect use case for Angular’s currency pipe:

this.items.forEach(item => {
    item['currentShares'] = this.currencyPipe
        .transform(item['currentShares'], 'USD', false, '4.2-2')
        .slice(3);

Again- we are doing a good deal of manipulating back and forth, so just using the easy route of piping in the template won’t work.  The code above loops over each item in the array and converts the ‘currentShares’ value to display as currency.

One small note here: shares aren’t actually currency, so we don’t want the ‘$’ or ‘USD’ to display, but the currency pipe is still very useful for adding commas.  We can just .slice(3) at the end of the chain to remove the ‘USD’ (the first 3 letters) from the string the pipe returns.  I didn’t find a built in method with the pipe to have no currency type display (it can display letters or a symbol, depending on the 2nd parameter passed), but this works quite well.

So far so good.  Our output is now showing as 1,021,400.00 in the grid (the two decimals are for consistency in display).  All done- maybe I can go home early!

But what about the sort?  Again- we’ve converted our number to a string.  In my previous post, we solved this by checking if: item[‘currentShares’] == +item[‘currentShares’] – but this won’t work here.  The string version has commas now- the above check returns false.

So a check was made at the call site of the orderBy function.  Before our array is sent off to the main method (in a shared service) to do the actual ordering, we check if the param (the property we want to sort by) is ‘currentShares’.  If so, we set a flag that is initialized to false (sortingByCurrentShares) at the top of our method over to true (more on that below) and loop through the items, stripping commas:

if(param === 'currentShares') {
    sortingByCurrentShares = true;
    itemsToOrder.forEach(item => {
        item['currentShares'] = item['currentShares'].replace(/,/gi, '');
    });
}

A super simple regex (the only kind I understand or trust) removes commas and replaces with nothing.  Side note: if you’re as hopeless with regex as I am, test them here – it’s a super useful tool!

Now we’re good on the sort.  The string version we are passing will “equal” (using the == operator) the number version and a numerical sort will work.  But now we need to revert to the string for output back to the grid.  In comes the sortingByCurrentShares boolean flag we flipped earlier:

if(sortingByCurrentShares) {
    this.items.forEach(item => {
        item['currentShares'] = this.currencyPipe
            .transform(item['currentShares'], 'USD', false, '4.2-2')
            .slice(3);
        });
}

The same basic logic as our original conversion (meaning I can probably move this out to a shared method- more fun refactoring ahead) is applied to convert back to the proper display string.  The user gets to see commas in their numbers.  The sort method gets to order numerically.  I get to check off another item in my task list.  Everyone is happy!

The Formatting Dance is Your Chance to Write Some Bugs

I love Javascript- but formatting numbers can be tricky.

Take money for example.  The Angular tracking application we’re working on has a grid to display data- and some of the columns contain dollar totals.  So, in our ngFor loop, we can just output the number in most cases.  You could even tack a dollar sign on the front: ${{data.money}}.  Works great if the amount is 19.99, but there are no trailing zeros allowed in numbers in JS, so if the amount is 20.00, the last two zeros are trimmed, leaving you with 20.

Not a big deal, but not exactly what people expect to see either- making the grid easy to scan and reducing confusion is very important in this app.  Not a problem- Angular comes with pipes.  It even has a built in pipe for currency.  Just pass your data, a ‘pipe’ character, and some config options in your template and you’re good to go:

{{data.money | currency:’USD’:true:’1.2-2′}}

That would tell your template it’s US dollars, show the $ character, and display at least 1 number to the left of the decimal and at least 2 but no more than 2 numbers to the right (this will round your number so be careful!).

Great!  Now it’s displaying as one would expect.  But we also have a search function on the grid.  Type in the input box and see your grid live update- thanks Angular magic!  But with a pipe, the data being displayed doesn’t match the data being searched through.  What was a number (5) is now a string (‘5.00’).  Search for ‘5’ and it works great, but a user may see the rest of the string and try searching for ‘5.00’- which will not work in this case.  No results found and a confused user.

Luckily, the awesome folks at Angular gave us an option for that as well.  Pipes can be used in a component instead of in a template.  Just import the pipe(s) you want to use at the top of your component file, inject them in your controller, and you can pipe away right in your methods.  It’s not quite as easy as just putting the logic in the template, but is a great option in this case.  We created a formatDataForSearch method:

formatDataForSearch(data) {
    data.map(d => {
        //loop through and convert specific items that need it- others remain as-is
        d.rate = this.numberPipe.transform(d.rate, '1.4-4');
        d.originalCost = this.numberPipe.transform(d.originalCost, '1.2-2');
        d.costBasis = this.numberPipe.transform(d.costBasis, '1.2-2');
    });
}

Works great- the data is formatted in the array that will be passed to our search function and everyone’s happy.  A user can type anything they see in the grid and search will find it, but the actual info stored in the database is the correct number type.

But wait- there’s one more problem.  We also have a sort function on the grid.  Click any table header and the grid live-sorts using that data.  I’ve chronicled my journey from buggy, terrible sort to less buggy, mediocre sort in a few blog posts, but the actual function is at a point where it works pretty well.  However, a string sort does not do the same thing as a number sort.  Pass [100, 2] to a sorting function and you’ll get [2, 100] (assuming an ascending order sort).  Pass [‘100’, ‘2’] to a sorting function and you’ll get [‘100’, ‘2’].  Why?  According to MDN’s entry on Array.prototype.sort(), “The default sort order is according to string Unicode code points”.

This means two things for our use case
1) If you don’t pass any comparison function to .sort, you’re probably ok for strings, but will not get the correct order for numbers.  Be sure to pass at least a simple comparison (a > b) will do for simple numbers.
2) Even with #1 covered, we will need to convert the data back to a number.  Otherwise, 100 will come before 2 in our grid and chaos reigns!

So, one more edit to our orderby.service.ts (which houses our sorting logic) was in order (ha).  And it gave me one of the very few legitimate (I think) use cases for == I’ve had in a couple years of writing Javascript.  We added a check in our sorting method’s data formatting logic:

if(a[param] == +a[param]) {
    first = +a[param];
    second = +b[param];
}

The first and second variables are initialized a little further up in the code- this conditional comes in a block checking the type of the data being sorted.  a and b are the things being compared in each loop of the .sort function (objects- with the ‘param’ variable being the title text of the clicked column in the grid).  So, if a comparison of the data in the sort column and that same data converted to a number (the ‘+’ operator is just a quick way of converting a number to a string) is truthy, we convert both back to numbers and compare.  The original display remains a string, so we keep those padded zeros in $5.00, but the grid sorts properly.

I’ve always avoided the double equals check, as it doesn’t really do a true comparison.  It does some type coercion and can give you unexpected results (0 == false, for example).  It’s generally safer to use the strict comparison operator (===) but type coercion does seem to have it’s use case.

What started off as a simple edit request in my job queue (display this number data in this specific way) became a rabbit hole of conversions and back again. To be honest, it was pretty fun.  And the journey wasn’t quite over.  I was feeling pretty good after this victory (however small), when I got an update on this job: “The current shares column should be formatted with commas.  For example: 2,004,100.00.”

Next week- the saga continues!  I can use the currency pipe to convert and add the commas, but those commas are going to break my nice x == +x check in the sort function.  Spoiler: my solution is ugly, but it does work- stay tuned!

The Refactor Factor 2: The Rerefactoring

A while ago, I wrote part one of this saga.  I can’t remember what it was about, and am too lazy to look it up, but the theme was refactoring.  I went back to some working code and made it (arguably) better.  At least slightly less crappy.  Today, I did it again.

The aspect of our application under the knife was the sorting function.  Most of our views are just table type grids of data.  These grids have headings (name, address, admin status, etc).  Click the heading and the grid sorts according to that criteria.

I’d created the sorting logic- generally pretty easy, I mean, Javascript has a .sort method built into Array.prototype- but there were complications.  The function would not know what type of data it would be sorting, so we will have to have branches for string and date and others.  Nor would it know the parameter(s) it would be sorting by.  It would have to toggle ascending vs descending sort (click the header once for asc, again for desc).  It would have to be pretty flexible with its inputs.

And- to be added currently, hence the need for the refactoring- the ability for a second sort criteria.  We found that with only one sort criteria (for example: name), the function worked great- as long as everything was unique.  However, what about a column like ‘admin status’.  That’s just a boolean field- only two possible values.  If a user clicked to sort by ‘admin status’, they’d get one listing the first time, but then might get a totally different listing on subsequent clicks.

To make a more stable sort order, we added a defaultOrder variable on each grid component’s class.  That defaultOrder would serve as the initial sort (when the view first loads), and be passed to the sorting service as the second criteria.  So, if a user sorts by admin status, the secondary sorting is done by the defaultOrder (example: name).  The output is consistent and less confusing.

Example time- and please know that I know this is still a bit of a mess.  As I refactored, I started breaking some of the logic out into helper functions, but found that this was creating more work and complexity than simply taking care of it within the .sort call.  I may take another look, but sometimes it just makes sense to write the code in a ‘top down’ type layout- though I know this isn’t often the case.

Version 1: A no good dirty mess (that still kind of works)

orderByParamAscending(param, group) {
    return group.sort((a, b): any => {
        if (typeof a[param] === 'number') {
        //numbers- simple subtraction sort
            return a[param] - b[param];
        } else if(param.indexOf('date') !== -1 || param.indexOf('time') !== -1) {
            //date from string- convert to date obj and compare
            const first = newDate(a[param]);
            const second = newDate(b[param]);
            return +first - +second;
        } else if (typeof a[param] === 'string') {
            //strings- check for empty
            let first = '', second = '';
            //convert to lowercase and compare number code
            if(a[param]) {
                first = a[param].toLowerCase();
            }
            if(b[param]) {
                second = b[param].toLowerCase();
            }
            if(first < second) {
                return -1;
            } else if(second < first) {
                return 1;
            }
            return 0;
        } else {
            //other (usually bool)
            if(a[param] < b[param]) {
                return -1;
            } else if(b[param] < a[param]) {
                return 1;
            }
            return 0;
        }
    });
}

I know- not pretty.  We’re using Typescript, which is awesome, but for a while I couldn’t get this to work.  TS would throw an error about a function (.sort in this case) not having a unified return value.  And that’s because it’s a bad function.  But you can shut Typescript up by providing the ‘any’ return type for a function- just know that this is an almost surefire indicator of bad code.

So- how could I make it better?  Well- instead of doing all those checks on each parameter, I could create a couple variables that would hold the value- then update them based on the type.  Also, .sort works well if you return 1, -1, or 0 depending on the result of your comparison function, so once the values are properly parsed, we can compare and return one of those 3:

Version 2: Slightly simpler, but still no secondary sort

orderByParamAscending(param, group) {
    return group.sort((a, b) => {
        let first = a[param], second = b[param];

        //check if this is a datetime param or string and parse so we can sort numerically

        if(param.indexOf('date') !== -1 || param.indexOf('time') !== -1) {
            first = +newDate(a[param]);
            second = +newDate(b[param]);
        } else if(typeof a[param] === 'string') {
            first = a[param].toLowerCase();
            second = b[param].toLowerCase();
        }

        //parsing done- time to compare

        if(first < second) {
            return -1;
        } else if(second < first) {
            return 1;
        }
        return 0;
    });
}

A bit easier to follow- and we can lose the ‘any’ return type- we are now always returning a number.  We handle all the formatting of the comparators (the ‘first’ and ‘second’ variables) before doing any comparing.

But the proposed additional functionality still isn’t there.  I initially thought it would just be a case of calling the function twice on the same array- the first time with the default sort param (‘name’ for example) to make sure the data is in the right order at the outset, then call it again with the new param (‘admin status’) to re-order, with the original as a fallack.  But this does not work- the first call has no carry-over to the second (which makes sense, but I had hopes for a simple solution!).

And, in the end, the solution wasn’t that complicated- it was just testing to make sure the sort returns the right order that was a hassle.  The only real changes were to the arguments passed and to the sorting logic at the bottom of the function- nesting the secondary sort into the initial in case of a ‘tie’.

Version 3: A little more complex than V2, but a little more functionality too!

orderByParamAscending(param, group, altParam) {
    return group.sort((a, b) => {
        let first = a[param], second = b[param];
        let altFirst = a[altParam], altSecond = b[altParam];
       

        //check if this is a datetime param or string and parse so we can sort numerically

        if(param.indexOf('date') !== -1 || param.indexOf('time') !== -1) {
            first = +newDate(a[param]);
            second = +newDate(b[param]);
        } else if(typeof a[param] === 'string') {
            first = a[param].toLowerCase();
            second = b[param].toLowerCase();
        }
        //do the same for the secondary sort parameter
        if(altParam.indexOf('date') !== -1 || altParam.indexOf('time') !== -1) {
            altFirst = +newDate(a[altParam]);
            altSecond = +newDate(b[altParam]);
        } else if(typeof a[altParam] === 'string') {
            altFirst = a[altParam].toLowerCase();
            altSecond = b[altParam].toLowerCase();
        }

        //parsing done- time to compare

        if(first < second) {
            return -1;
        } else if(second < first) {
            return 1;
        } else {
          if(altFirst < altSecond) {
               return -1;
          } else if(altSecond < altFirst) {
               return 1;
          }
          return 0;
    });
}

There are other aspects we added- a ‘sortDirection’ toggle, a check for null values passed to the sort to convert them to strings (the comparison was failing on null- but some values from the database were null), but this is the meat of the function.  And I’m sure there are still improvements to be made- programming seems a little more art than science sometimes.  It can be hard to tell when you’re ‘done’ with something.  I was ‘done’ on version 1, but version 3 is much better.  At some point, you just have to move on to the next task, or your product will never ship!

Pass the reference, please

The current landscape of Javascript, with all the fancy front end frameworks, is great.  It allows for rapid development of really cool applications.  But it’s important not to forget the basics.

On our Angular application, we get data from a database on many actions (shocking- I know).  To normalize the process (making sure the right authentication is passed with each request, that the data returned is in the proper format, etc), we created a service- “backendService”.  This wraps around Angular’s http module, so it returns an Observable to the component that calls to it.

Generally, that observable provides an array of objects- let’s say “companies” in this case.  The company will have a name, location, number of users, etc.  When our backendService fetches the company list, there are a few tasks we need to do with the data.

  1. There is a d3-driven chart for the top 5 companies by user count (the 5 companies with the most users).
  2. There is a drop down to jump to a list of that company’s users.

For task 1, we manipulate the data into the top 5 by user count and sort accordingly (most users to least within those 5).  For task 2, we just sort alphabetically by company name.

Experienced JS developers are probably already seeing where I’m going with this.  And I should have seen it too- but theoretically knowing that an original array is modified when a shallow copy of that array is modified is a bit different from managing that process in practice.

In the subscription to the data, we had simply assigned the response to the class ‘companies’ variable (an array initialized with no items) so it could be passed around to other methods and even other components/services.  Something like this (general idea only):

this.companies = []; //class variable
this.backendService.getData(urlBaseHere).subscribe(response => this.companies = response);

Works fine- as long as you don’t need that ‘response’ array for anything else.  When we added the d3 graphs, we (I) forgot to go back and check the source.  We just added a service to filter and sort this.companies to get the top 5 by number of users.  It wasn’t even in the same JS file.  It’s generally best practice to break out your code into small, reusable components, but there are some drawbacks, particularly when working on a team.  If you’re sharing a reference to an array, one file may manipulate it one way while another is trying to do something different.

So, when we added the chart code (order by number of users), I found that I couldn’t sort the drop down alphabetically anymore).  Initially, I blamed the back end.  I started on a PHP/MySQL stack, so I know a bit of SQL, but this project uses SQLserver- the Microsoft flavor, and I’m less experienced with that specific brand.  I thought maybe the default sort I put on it (ORDER BY company_name) was flawed.  I like working on the backend, but with a language like C# (that is, a compiled language), it’s a bit slower development process.  Make a change, kill the dev server, re-compile, and restart the server.  I know- it’s not a big deal, but again, the cool frontend frameworks have spoiled me with their ‘hot reloading’ and ‘automate all the things’ awesomeness.

Anyway, after wasting the better part of an hour researching MsSql syntax and discovering it was not the culprit, I went back and double checked the component getting the actual data.  When I logged the ‘response’ parameter directly, I noticed that it was not giving me the array sorted by company_name- it was giving me the array sorted by number of users, not alphabetically by company_name.  I’d manipulated this.companies and therefore manipulated the base array returned.

The answer was easy- though there are a few options we could have gone with.  We just initialized two arrays: this.companies and this.companiesForChart.  Then, we assigned both to the response before doing any manipulation.

this.companies = response.map(company => company);
this.companiesForChart = response.map(company => company);

Using .map ensures that each is a deep copy of the array- not just a copy of the reference, and also reminds anyone else using the code to use the same process (and that they can do any manipulations there as well). Each one gets a reference to the original data, but this.companiesForChart is free to do it’s own thing and not mess with the default sorting (alpha).  It’s passed off to our chart service file where it’s .filter and .sorted as much as we need. Another option would be to use the slice method:

this.companies = response.slice(0);

Although we didn’t test this- I think it might also cause issues when we manipulate this.companies, so the need to have the second array (this.companiesForChart) would remain – also assigned to response.slice(0) in this case. Either way, it made a bit more sense to me to have two copies of the array, as there were two tasks and two different files using those arrays. It might be a little slower to initialize 2 arrays (not sure on that), but for other people on the team looking at the code, this makes it a bit more explicit exactly what each one is for.

So- the moral here is: don’t forget the basics.  When I logged the ‘response’ to the console (right after it was retrieved from the backend) and saw the array ordered by number of users, I assumed the problem was with the backend sorting.  How could anything in my front end code manipulate the array that just came back?  But the nature of JS arrays was the real issue- actually, me forgetting about reference types vs value types was the real issue.

When in Doubt, Log it Out

I still have a hard time judging the schedule for some projects.  I know this is a matter of experience: if you’ve set up a basic Angular application, you can probably give a solid timeframe for doing it again.  But given a new task, it might take 30 minutes or 3 days.

And sometimes the little things can make the difference between the two.

A couple weeks back, I’d hit a roadblock in our new Angular project (related to a piece of 3rd party software that I just couldn’t get to run reliably on a test machine).  To take a break, I started playing with d3 with the idea of hooking up the data source as an observable to have a ‘live refresh’ pie chart anytime someone adds a new user to a specific group.

Before integrating an observable in the mix, I started just setting it up with a returned array of the top 5 results from the database.  Seemed to work- but I also wanted to add hover effects.  Ideally, hovering over a piece of the pie, or hovering over an entry in the legend (showing the name of the group and their total users) should change the fill color of both.

That was a bit more tricky- my first thought was that d3 has a jQuery like syntax, so maybe it has some kind of .find() selector.  Couldn’t get that to work, but I did find a great tip (lost the link to it, but if I ever find it again, will cite here as it was a really good idea) to use a data- attribute.  The tutorial with that tip just put the data-key attribute directly onto the svg element in the html, but I reasoned it would be easy to do with d3.  Just add a .attr(‘data-key’, (d, i) => d.company_id) to the chain on both the legend and pie generation code and it should pop right up.  Then I could d3.selectAll(`[data-key=’${key}]`) for that data-key attribute to get both the legend text and pie piece (which would share the same data) and add a .on(‘mouseover’) event handler.

And it was easy – for the legend text.  I added the code, checked the element in DevTools, and the data-key was right there with the correct company id.  But there was nothing on the pie piece path in the svg element.  There was no console error, and the chart displayed just fine, but no data-key attribute would appear.

I started tinkering.  Maybe the order of where you add the data- attribute in the d3 chain matters, so I tried first, last, and everywhere in between (nope).  Maybe there’s a special syntax for adding a data- attribute to a path element (nope- just the normal syntax).  Maybe you can’t add a data- attribute to an svg?  But I’d already done it successfully with the legend elements.

Finally, we get to the title of this episode.  In the .attr property function (remember that you can pass a simple value as the 2nd argument to .attr, or you can pass a function to get the data associated), I added a console.log call to see what I was getting.

The answer was: nothing.  I was getting undefined for my d.company_id property, even though it was working for the legend element.  The issue was that the path element (which defines each ‘piece’ in the pie) is further ‘up’ the chain: meaning that it doesn’t just get the data, it gets a larger object (I think it includes the positioning info as well).  The point is, there was no ‘company_id’ on the object I was passing, so I was actually trying to assign my data-key to ‘undefined’.  I would have guessed this would cause some kind of error, but it apparently doesn’t- it just fails silently.  No data- attribute gets added but nothing blows up either.

So- when in doubt, log it out!  I could have saved time by checking this.  The only real issue was that I had to dig a bit deeper into the object being passed to the ‘path’ element- it has a ‘data’ property with the info I needed, so I was looking for .attr(‘data-key’, (d, i) => d.data.company_id).  With that change, I could grab both elements in the svg with the same data-key and highlight both on hover.