Fat Model in a Skinny View

Working with Django can be pretty fun.  I’m not quite as familiar with the patterns and recommended practices as with Javascript and it’s associated frameworks, but there is a cool mantra I learned recently: Fat Models – Skinny Views.

I’ve been working on creating an API for a product website/blog.  The front end is latest Angular, the back end uses Django to return JSON (in as close to a REST API as we can get).  Because my Python’s a bit rusty (our latest project used .NET as a backend), I was catching up with some Pluralsight videos.

Side note: Pluralsight is a great resource for learning more about many programming languages, frameworks, best practices, and so on.  I’m not an employee there, just a satisfied customer.  The series I was using to brush up on Django was called Django Fundamentals.

Anyway, the instructor mentioned a phrase that resonated with me: Fat models and skinny views.  Basically, keep your logic in methods on your models, and make your views responsible only for returning data as much as possible.  Until this point, we’d been haphazardly filling up our views with all the logic and it was a bit of a mess.

The fat models/skinny views pattern allows for more abstraction and reusability.  Example time!

One of the views is to simply get the blog data in JSON format for display.  If the request passes a ‘slug’ (the identifying string for an individual blog), we get just that blog entry.  Otherwise, we get all.  Nothing super complicated- but joins on a database table can be tricky in Django’s ORM (the layer on top of any SQL transaction).  The ORM is useful, but I just couldn’t get it to join our main blogs table with the category table and the authors table.  As a result, the frontend was just getting the author id and category id instead of the actual string name/title (because only the id was stored in the blogs table- in a Foreign Key relationship).

From a bit of digging into Django, I found that you can ‘prefetch’ that related info using the select_related method.  So our view started as:

def blogs(request, slug=""):
    if request.method == 'GET':
        if slug != "":
            entry = Blog.objects.select_related('category', 'author').get(slug=slug)
            return JsonResponse(json.dumps({"result": True, "msg": "Blogs Loaded", "data": data}), safe=False)
        else:
            blog_list = Blog.objects.filter(posted__lte=datetime.today()).order_by('-posted').select_related('category', 'author')
            return JsonResponse(json.dumps({"result": True, "msg": "Current Blogs Loaded", "data": data}), safe=False)

We check the request (only GET is allowed on this unprotected route- there’s a different one with JWT auth for updating/adding entries) and then check for existence of a slug.  If one is passed, get the individual entry with the related info prefetched.

But there was an issue.  On the frontend, we logged the JSON response, but no author or category info was included.  Just the data from the blog table.  I went back to the Django side and printed out my “entry” and “blog_list” variables- everything was there!  From a little more research, it seems that the above method works just fine if you’re using the default Django view template, but we weren’t- we just wanted JSON data passed to our Angular components for display.  It looks like when the entry/blog_list collection was converted to JSON, only the original object info was coming along for the ride.

But the related data (category and author) was available on the original entry/blog_list object.  So, we could just loop through and grab all the properties we need, add them to a simple Python dict, and return that.  With that solution, our view officially passed from slightly chubby into full on fat.  The loop logic was repeated in multiple places and multiple views- not good for future developers who might have to update this code (probably me!).

So, we made a slight update.  As an example, for the above code, just after we get the Blog.objects collection (either the individual blog or group of blogs), we call to a method on the model:

data = Blog.get_blog_and_join(entry)

That one’s for the individual entry. On the model, we created a new method to handle that logic:

def get_blog_and_join(entry):
    data = {"pk": entry.id, "title": entry.title, "slug": entry.slug, "author": entry.author.username, "body": entry.body, "posted": entry.posted.strftime("%Y-%m-%d")}
    if entry.category:
        data["category_name"] = entry.category.title
        data["category"] = entry.category.id
    else:
        data["category_name"] = "No Category"
    return data

This just loops through the entry passed (the Blog object returned by Django’s ORM with related author and category data) and assigns the values to properties in a new dict.  Note the author prop- it references entry.author.username.  This didn’t work in the original method of just returning a JSON encoded version of the Blog.Object returned by the ORM.  A little extra logic to check if a category exists (they’re optional in our setup), and now we have a simple object with all the info we need.  It can be returned as JSON and the front end team has their precious data!

Advertisements

A Greater Datetime

Django’s built in blog module is very cool.  Go back a few months’ entries on this blog and you’ll see that’s where I learned Django (and how most people do, I would think).  Powerful, well-documented, easy to extend- just a great thing to have available.

One of the features most blogs have these days is the ‘publish later’ options.  You can pre-create a bunch of entries, then have them post on certain days.  So it looks like I’m at my desk, being productive, while in reality I’m sitting on the beach, drink in hand.

Publish later isn’t built in to the default Django blog module, but it is pretty straightforward to add.  The default module does have a ‘date’ field- which allows you to set the date attached to the blog.  Why not use that?  Just update the view for your blog display to filter by the date field (set the filter to only show blogs with a date < today:

Blog.objects.filter(posted__lte=datetime.today()).order_by('-posted')

This also makes use of cool aspects of Django’s ORM- just throw a double underscore and ‘lte’ on a field and you have a SQL condition for ‘less that or equal to’.

There you go.  When creating a new blog entry, just set the date field (with built in calendar picker) to the date you want it to publish.  The entry is saved, but won’t display on your front end until the date you selected.

Simple?  Yes.  Impressive?  Probably not, but it was quick to set up and it works, so that’s good enough for me!

Drowning in a Sea of Dead Links

So there was a problem.  Shocking, I know.

The new site development is humming along (mostly)- and might be ready for launch soon.  The problem is the old url links.  The old site was not created using any kind of framework, it was just simple static html pages.  The new site has multiple frameworks that need to play nice together.  That’s not a problem, except with search engine optimization.  Apparently (and I’m no SEO expert), changing all your url routing can negatively affect your search results.  Links will be found that go to old, dead pages- or worse, nowhere.  Frameworks like Django and Angular don’t use the ‘.html’ suffix generally, so this could mean having to create a bunch of server url redirects.

Which gave me an idea.  Is it a good one?  Not sure, but it did seem easier and more efficient than a whole bunch of server redirects.

It goes like this: What is the Django urls.py file but a kind of layer on top of your web server?  It takes an incoming url string, matches it against a regex, and passes it along to the view.  So, why can’t we use that to redirect our old urls to the new method?

The answer is: we can!  One simple Django url setup:

url(r'^(P<path>[a-zA-Z0-9_-]+).html$', old_url_redirects)

And a slightly more complicated view setup (condensed for brevity’s sake):

def old_url_redirects(request, path):
    if path in {'new-route-one', 'new-route-two'}:
        category = 'new-middleman-here'
    return HttpResponseRedirect('/newfolder/'+category+'/'+path)

And there you have it!  The view function is a bit ugly because we reorganized the structure a bit to add categories into the url string (the old method just had one massive folder with all the endpoints).  If you don’t need that, you don’t need the ‘if path in’ conditional, and can remove the category variable from the return statement.  But the cool part is the ability to make a simple check over multiple options with a Python set.  So if someone gets the old link (http://www.mainsite.com/url-here.html), it will automatically redirect to the new route (http://www.mainsite.com/correct-category/url-here).

Does this method suck?  Maybe- but it does work!  If I’m missing something, or you see a more efficient way, help me out in the comments…

Grunt with the Effort

I was tasked to find a good service to concatenate/minify our javascript and css files.  Of course, I’d heard of these words before, but never actually used the techniques.  My other major project involved some very old practices, including inserting php directly into a css file (and rarely, a js file), so minifying would absolutely not work.

But I’d heard of a couple- Gulp and Grunt for example- so I took on the project.  I checked out Grunt first, for that time-honored reason: the alphabet.

And it’s awesome.  It was a little intimidating at first, but after going through the initial setup (pretty easy with npm) and taking a little bit to learn about the two core files (package.json and the gruntfile), I found it to be really friendly to work with.  After all, it’s really just a bunch of javascript, and I like working with that!

The stumbling would come later, when I realized that our new project was built in Angular JS by many different developers.  With no current js or css style guide, I started to run into trouble.  The minifier doesn’t like Angular’s syntax on some things, particularly templateUrl within a directive, but setting the ‘mangle’ config option to false seemed to help.

Minifying was working, but concatenating is still not.  There are errors or unclosed tags somewhere in a directive (not written by me!  I never make mistakes- really boss!).  Somehow, they work in a full sized file, but it does not survive the minification process.  So now it’s time to check each directive, and each template html file to see where we’ve gone astray.  Not looking forward to that.

Speaking of things I’m not looking forward to: we’re trying to push the latest version of the Angular-Django app from our local machine (running the default Django dev server- where it runs perfectly) to the remove dev testing server (where it doesn’t run at all).  This also looks to be an issue with Angular- the templateUrl isn’t getting the right path, which is throwing a very unhelpful error in the console and blocking almost everything from loading.  I know hunting bugs is a major part of the job, but it might be my least favorite.

It has, however, convinced me of the need for a style guide.  It’s not really up to me, but it will be suggested at our next meeting.  There are plenty of good ones out there to emulate (copy).

Non-admin admin

I’ve been working with a team for the first time on a project, and it’s been an interesting experience.  I’m still at the steep point on the learning curve, so it’s tough to know if my thoughts and instincts about some tasks are legitimate.

An example: We are redesigning both a large scale website, it’s admin panel, and a blog app within the website.  We’re moving to a Django powered site/blog with Angular as our front end framework.  The higher-ups on the team have decided that the admin panel should be a single page app.  I know these are all the rage, and when well-done, are really easy to use, but it seems unnecessary for some aspects of the admin panel.  Django comes with so much built into it’s default admin panel.  We used it to create the entire blog app- it handles the routing, and comes with the basics of an admin panel.  It also has a great user management system baked in.

It’s not as cool as a fully single page app would be, but it seems like a waste to junk so many of the built in features.  So I’ve been trying to figure out a way to import the Django admin features into a normal Django view.  The models seem to be the easy part- but so much of the url routing is built in, it’s going to be tricky to try to move that to something like a $http routing structure that would be needed with a single page app.

My other hope is that we won’t have to change the front end routing of the blog app.  But if we change everything about the admin side, it seems like a pipe dream that the front end would magically work.  The front end is ready to ship, so I’m keeping my fingers crossed for that magic!

And maybe none of this makes sense.  I hope that’s the case- if so, I will learn some really cool stuff about manipulating frameworks to meet specific, custom needs.  I read a few blogs/tutorials about integrating Angular with Django, and they make good arguments, but none seem to use the built in admin.py file.  At least not without quite a bit of difficulty.  It seems that using both takes a little away from each: the front end framework (Angular) wants to do much of the work that the back end framework (Django) already does (and vice versa).

Placeholder Workaround

We’ve still been working with Django on a redesign of a site’s blog module.  Still liking the tool- it’s a bit tricky to figure out initially (the url routing in urls.py took me way too long to wrap my head around- and I wouldn’t say I’m fully clear on most of it), but overall it’s so useful.

But in any bargain, there are downsides.  We were customizing the log in screen to get away from the default Django scheme (to try not to scream “We used a template for this!”).  Got the template overriding process down, but ran into an issue on something that should have been real simple.

We wanted to add placeholder text to the username/password fields (we were trying to create a slick, no-label login form).  But the actual input fields are generated ‘behind the scenes’ by Django.  Usually very useful, as then we don’t have to do it ourselves, but if there’s something it doesn’t include by default (like placeholder text), it can be tough to add it in (at least for a Python newbie like me).

I did some Googling and saw some smarter folks than I found a way to do it directly in the model (though it did look more complicated when using Django’s default admin panel log in), but my experience is mostly in Javascript, so that’s what I used.  It was really simple once I thought about it properly- just a call to the input field’s “placeholder” attribute to set it to “username” and “password” and we were good to go!

Sometimes it’s best to fall back on your strengths (at least, that’s what they tell me- and I tell myself JS is my strength!).

Things to Remember

Still really liking Django (and by extension, Python).  Some things are a bit tricky to figure out, but overall, it’s really easy to use and can be a very powerful tool for developing on the web.

Deploying to a live server can be one of those tricky areas.  Django makes it super easy to build locally, including a server with it’s package.  However, eventually, you’ll want to put your creation somewhere on that world wide part of the web.  The project I’m working on already had a server available, so I FTP’d all the files to the right directory, remote logged in, hit my manage.py runserver command, and opened my browser to check it out.

And saw a debugging error telling me there was an invalid object.  I’d become somewhat familiar with the error messages Django throws during the initial development, but this one had me stumped.  I checked the schema- all looked right.  I double checked my models and they looked fine.  I’d even installed South (the project requires Django 1.6- don’t ask me why), but it couldn’t find any issues.

Then I realized that I’d tested this all locally, but never actually run an initial migration on the remote server.  The error was telling me there was an invalid object because the view was trying to access an object that simply hadn’t been initialized.  I ran a syncdb command on the remote server and everything started working!

So, I guess the moral is to try not to forget those first steps.  I hadn’t really checked anything locally when first setting up the project- I hadn’t opened the site in a browser into quite a way into the initial process, so hadn’t seen this issue.  I went back and typed out a startup guide so I wouldn’t forget.  Eventually, I’ll remember these things automatically, but for the meantime, it’s always good to have a backup.