Pulling the strings

So, back to the puppet show.

Actually, I guess this is going to be a bit more of a tangent, but it did come up when working on the new Puppeteer project (to crawl our Angular application and store server rendered pages in a local cache), so it still counts…

We wanted to have both a command line option and a graphic UI option to run the crawl service.  The UI would need a backend that keeps a websocket open and broadcast out updates when a new url was crawled (to show success or failure, and give stats at the end).  Socket.io works great for this- just install it on your Node project, and you can socket.emit messages and data to your frontend, which can listen with socket.on (use the same name for both emit and on to coordinate the two).

However, in the command line option, there would be no socket.  With this configuration, a user would just run the command and the messages/data should print to the console.  So we have a shared program that needs two different interfaces.  I had already created a “crawl-runner.js” file with my main “run” function.  It would handle the browser init, page creation, and navigation in headless Chrome (using Puppeteer).  It also handles storing and responding with the results.  It was set up to use a simple native Node EventEmitter- which worked fine for interfacing with websockets.  In fact, we could probably just cut out the middleman and eliminate the EventEmitter- just socket.emit directly from the crawler.

But either way, we will have to switch to console.log when using the command line option.  How to reuse the logic from crawl-runner.js in the command line version?  We can pass the emitter as an optional argument to “run” and if it’s not there, alias that name to console.log:

When the program is run in interactive, UI mode (via a dashboard on our Angular app), crawlEmitter is passed to run, and the socket interface works.  When it’s run as a command line application, we still call “crawlEmitter.emit” with the message and data we want to send, but the check at the top of the function will call “console.log” whenever “crawlEmitter.emit” is called (because there is no crawlEmitter in this case).

Another option would to be simply passing the function we want to use as a broadcaster into run.  So, pass crawlEmitter.emit as the 2nd argument for the dashboard version, or console.log for the command line version.  That might be a better, more readable solution, so I’m thinking about switching (haven’t tested this yet- but I don’t see any reason it shouldn’t work).

One of the most fun things about programming is how many roads you can take to one final product.  The trick is finding the balance between most efficient and most understandable – and always being open to finding a new route!


Form Autofill Follies

Been a bit lazy lately, but trying to get back in the groove of writing this thing.  Had a lot of failures lately, so let’s talk about one of those!

I was working to redesign our login screen.  It works fine, but is very basic.  Time to add some jQuery flash!

The idea was to absoultely position the form labels (username and password) over the input fields (which were styled to appear just as underlines).  When a user clicks on a field (or focuses there via tab), the label animates to the right of the input field and shrinks down.  If the input blurs, there are two options: 1) if the user entered something, the label stays small and out of the way.  2) if the user didn’t enter anything, the label grows and moves back into place.

And it worked great!  Took some tweaking to work properly, but I thought it was a cool little trick.  But in my testing, I finally used a browser that had stored credentials for the login screen and everything fell apart.

Different browsers have different events when they autofill fields, and it seems none really broadcast those events to hook into.  As a result, if a user stored their credentials, they were greeted with text on text (their credentials, overlaid with the input labels).  The focus events didn’t fire.

So I scrapped all that and just went with a simple setup of labels over basic input fields with some useful placeholder text.  It’s really fun to create cool looking interfaces, and when it finally works, I always get a rush from the accomplishment.  But it’s important to remember that users don’t care about your rush.  They just want to log in to the damn site, so make it as easy as possible!

If you’re curious about what might have been, I cooked up a quick codepen to show a general idea.  Check it out at http://codepen.io/joedriscoll79/pen/NNoGrv

Feel free to tell me how ugly it is- I though it was kind of cool.  Have an idea for how it might work better (or at all with autofill)?  Let’s discuss!


Query Queries

The recordkeeping project is getting a bit out of hand.  It’s fun to work on, but if there had ever been an original scope of project (doubtful), it’s been left in the dust long ago.

100% my fault- I’ll be working on the approval tracking option and realize; “This really needs email notification to be truly useful.”  Completely true- no one wants to log into the app each time they need to check if there’s been a response- they should be able to get email updates.

But it shouldn’t be required, so now it has to be an option.  And who should get the email updates?  Everyone involved?  Just those with posting auth? A small addition snowballs into a couple weeks of work.

One of the tricky parts of finishing the approval option was the SQL involved.  The original feature had no tracking feature- there was basically just one large textarea field where posters could all add text/edit any existing text.  So the design of that db table was very simple.  But in adding both individualized entries, and tracking individual approval recordings, we added quite a bit of complexity on the db front.

Not something I have a ton of experience with, so there were some false starts.  Making sure each poster’s approval setting was unique depending on the recordkeeping category entry required a few joins, but we got there eventually.

All that work done, and we decided that the main entry screen should really be more of a timeline- tracking normal comments, file uploads, and approval entries.  It’s tough when a mid-project redesign has to happen, but sometimes it really is for the best- this was a good idea and fixing it halfway through made a lot more sense than finishing and shipping the project, only to promise another update later (which might never happen- there are other areas to improve!).

So there were some good lessons learned regarding database design.  And some frustrating lessons learned regarding syncing the UI and the backend.  Makes me appreciate the new JS frameworks that do a lot of that work for you- and do it much better than I am!

More user shenanagins

Did I spell shenenagins right?  Probably not.

More work done on reorganizing and upgrading the user account creation screen for the CMS I’m working on.  ‘Above the fold’ is now the welcome message, help link, and basic user info (name, username, password, contact info, notification option).  Previously, the email info entry had been at the bottom- caused a good deal of confusion.

Then there are save options.  Setting up a new user requires an initial save after entering the basics, so this seemed like a good idea.

After that are the other options.  Home/directory/email list config.  Admin authorization options, etc.  These are much less vital (though home assignment is an important step).  Need to do a little sql magic to get the current info from the db for some items (directory privacy settings, current email lists subscribed to).  Then we will work out the rest of the client side js checks on the form itself.