A Nomenclature for Musicality

I think that one of the big impediments to learning musicality is that many people lack the language to verbalize music. Over the years I’ve picked up various techniques and terms for this that help me to think about the structure of music, talk about it, and yes, even dance to it.

So to aid other dancers who may not yet be familiar with these very useful musical terms: here they are.

Tempo

Tempo is how “fast” a song is – in other words, the number of beats of music per minute (BPM.) This is an easy one, since as a dancer it also determines how many steps you will take per minute. Importantly, it is possible for a song with a fast tempo to nonetheless be pretty low energy, and conversely songs with a slow tempo can be high energy (think dubstep.)

This relates to dance since we have to dance differently to fast songs than slow songs. In particular, to fast songs you have more movements to complete in less time, so you can accomplish this by taking smaller steps and making smaller movements overall.

For my Westie friends, the standard BPM range for West Coast Swing songs is between about 80 and 120 BPM.

Saying a Rhythm Aloud

This is actually a surprisingly important skill across multiple dance disciplines. Ever wonder if a song is a Rumba or a Cha-cha? The rhythm is what makes the difference.

There’s volumes about this in music theory, but as dancers the skill we really want is the ability to count sixteenth notes, since that is about as fast as you can dance in a partner dance.

Most of us will be familiar with counting quarter notes: “one two three four.” And most dancers also know that they can stick an “and” halfway between the numbers to count eighth notes: “one and two and three and four and.” But did you know that you can also take this a step further and count the notes halfway between the “and” and the numbered notes?

It goes like this: “one-e-and-a-two-e-and-a-three-e-and-a-four-e-and-a”. For this concept, sheet music can help to visualize it, and this video gives an excellent overview:

So why does this matter? Well, in ballroom, the rhythm is what makes a song “be” a certain type of dance. As one example: cha-chas all have the rhythm “one two three four-and-one.” The “cha-cha-cha” of the dance falls on the “four-and-one” triplet in the song’s major rhythm. Similarly, Sambas are danced to songs with a “one a-two three a-four” rhythm.

And since I’m a Westie, you probably guessed that this would eventually relate back to West Coast Swing. A common term we hear in the WCS world is “Blues Timing” or “rolling count” – but some people aren’t familiar with what this actually IS.

Since we now have a way of describing rhythm, we can describe it as well: Blues Timing is when we change the timing of our triple steps by taking the second step (which would normally fall on the “and” beat) a tad later later, on the “a” beat. We use this and other techniques such as kick-ball-changes in order to represent music which is itself emphasizing the “a” beats, as in: “a-one a-two a-three a-four.”

In WCS this is one of the defining characteristics of the blues songs we hear in competition, so it’s an important technique to know! They aren’t playing the blues songs “for the old people” as a friend put it once – they’re playing them so the judges can see if you know Blues Timing and blues phrasing (covered below.)

As one example, listen to “Wild Turkey 101 Proof” by Kenny Wayne and try counting it “one and two and three and four and” – you’ll find that it just doesn’t feel quite right because of the emphasis of the bass guitar on the “a” sixteenth notes, rather than the “and” beats that are more prominent in contemporary music. Then try counting it “a-one a-two a-three a-four” and you’ll find it fits the music much better:

Staccato versus Legato

Staccato and legato are musical terms which describe the character of a given note. In particular, it describes whether a given note in the music is isolated from the notes that come before and after it, or whether it flows smoothly into the next note.

An example of a staccato note in a song would be a drum hit, or clapping sound. After it occurs, there’s a period of silence before the next note by the same instrument. Conversely a legato sound would be something like a long violin note or a singer singing a melody without pronounced pauses between the notes.

These can be useful cues for the musicality in our dance. Short, Staccato notes lend themselves to body isolations and rapid changes in speed. Legato notes in turn lend themselves to longer, flowy movements without quick stops and starts. This is a great place to start on your musicality if you’re not sure how to dance to a song: simply find the most prominent instrument, and determine whether it is playing staccato or legato, then dance accordingly.

Measures/Bars

Another important thing to note is the number of beats in a phrase. Music is generally written in 4-beat measures (or bars.) That’s the same intervals we were counting above, and you’ll notice that it doesn’t go all the way to 8! This is one of the ways that dancers differ from musicians – musicians like to count to 4 since that’s how long a measure is, and how sheet music is structured. Dancers like to count to 8 because… well, we’re not usually done with our pattern after just 4 beats! We don’t want to start over in the middle!

This is the reason some top pros (Robert Royston is a prominent one) are advocating that dancers start counting in fours rather than in eights – it helps keep your mind on the music and put dancers back in sync with the music and musicians.

Time Signature

A song’s time signature is a quick way of describing how the song’s repetition is structured. The time signature of a piece is often written as a fraction, and the most common is 4/4 time. The top number refers to the number of notes in a bar/measure. While the bottom number represents the size of those notes. So, 4/4 time means: 4 beats to a measure, with each being a quarter note.

While 4/4 time is the most common time signature, all sorts of them exist. But other common ones include 3/4 for waltzes, and 6/8 for Viennese waltz.

Bonus: if you’re ever DJ’ing and want to really troll a room full of dancers, find a song in 7/8 time and put it on at the end of the night. Everyone will be off time and think they’ve suddenly forgotten how to count music!

Phrasing

Phrasing is another of those terms that is thrown around a lot, but often not actually defined. Simply put a phrase is a complete unit of repetition, and could be played as a self-contained musical segment without sounding “incomplete.” Importantly, when a phrase ends, it is very common to change the music in some way, by increasing or decreasing the song’s energy level, changing the lyrical structure, or adding/removing one or more instruments to the piece.

As dancers, phrasing is crucial. By definition, music is repetitive – without repetition, music is just called “noise.” But the phrase changes are the moments where something new is introduced to the song – and as dancers we aim to express the song through our bodies, so it’s important that we introduce something new to our dancing as well, to reflect the song we’re dancing to.

In most modern music there are 8 measures to a phrase, for a total of 32 beats in a phrase. This means that you could actually count to 32 over and over and nail the phrase change every time! I don’t recommend this, since as we dance we need to be focusing on more than just counting in our heads… but this is a great exercise to do on your own while driving and listening to music!

It’s also important to know that in 12-bar blues there are 12 bars to a phrase! So as Westies when we dance to a lot of blues songs, there will actually be 12 bars of 4 beats each for a total of 48 beats in a phrase. As an example, try counting along with the first 48 beats (33 seconds) of Creeper Returns by Mark Hummel and notice how the last 8 beats of the phrase are a buildup before the end of the phrase:

Song Parts

Now that we know what a phrase actually is, we can start to talk about the song as a whole by giving names to the phrases! For instance, a typical song might look something like this:

  • Intro
  • First Verse
  • First Chorus
  • Second Verse
  • Second Chorus
  • Bridge
  • Outro

Let’s define these terms:

Intro

The intro is, put simply, the song’s introduction. It is often not repeated again later in the song, is musically distinct from the first verse, and may be anywhere from 16 to 64 beats long, depending on the song. A good example is the beginning of Secret by Maroon 5 – notice how much time in the song is spent even before a repeating melody is introduced, and how much more before the vocalist begins singing the first verse about 1:34 into the song:

As a dancer, the intro is an important opportunity to “groove” to the song before it’s really gotten started. For most songs, musically, it doesn’t make a whole lot of sense to start throwing dips and spins into our dancing. Rather we will usually want to keep it simple, focus on connecting to our partner, and getting ready for the first verse.

Verse

The verse is what could be thought of as the “main” part of the song. in songs with lyrics, this is where most of the song’s story is told. Verses are generally very similar to one another in terms of instrumentation, but the vocals will almost always be different. As an example, consider the lyrics for the first verse of Don’t by Ed Sheeran:

I met this girl late last year
She said, “Don’t you worry if I disappear.”
I told her I’m not really looking for another mistake
I called an old friend thinking that the trouble would wait

And compare this to the lyrics of the second verse:

And for a couple weeks I only wanna see her
We drink away the days with a take-away pizza
Before a text message was the only way to reach her
Now she’s staying at my place and loves the way I treat her

We can see that not only are they different words, but they also tend to make up chapters in the songwriter’s story. Bonus tip: the “knock knock knock on my hotel door” that you want to hit while you’re dancing is at the start of the third verse. This is why having terminology to describe these things is useful!

Chorus

The chorus generally contrasts with the verse by being higher energy, and containing lyrics which are repetitive – often both within a single chorus, and also between multiple choruses in the same song. This is also often where the song’s “hook” is in pop music – the catchy bit of music paired with the memorable lyrics that makes the song stand out in people’s minds. It’s also often where the lyrics that the song’s title comes from are sung.

Bridge

A song’s bridge is a part that comes in the middle of the song, and is often used as kind of a “break” from the repetition between verse-chorus-verse. This is often where instrumental solos will happen, and the bridge usually isn’t repeated.

A good example of this comes at 3:07 in the song Madness by Muse, as the third verse ends and the electric guitar comes in, carrying us into the fourth chorus (notice also that this song starts with a chorus instead of the first verse – this is why having names for things is useful!):

Pre-Chorus

Some songs will have a pre-chorus, sort of a “bonus” chorus – usually 16-32 beats that immediately precede each chorus, while having a slightly lower energy level than the chorus, while still being higher energy than the main verse. These provide sort of a “build up” or transition between the verse and chorus, and are usually repeated before each chorus of the song.

One good example of a pre-chorus is in the song Gibberish by MAX. Notice how the first verse wraps up about 28 seconds into the song, then the pre-chorus slowly raises the energy level for 32 beats before the chorus kicks off at 51 seconds in. You’ll also hear the pre-chorus again before the second chorus starting at 1:25 in:

Outro

Just do a dip or whatever.

Just kidding, but the outro is pretty self-explanatory – it’s a piece of music that, like the intro, is not repeated anywhere else in the song, but comes at the end. Not all songs have outros – some just end after a verse or chorus is done, but others do, so it’s nice to have a term for it.

Conclusion

Music, as with all forms of art, can’t always be boiled down to categories we can put names to. There’s exceptions to each of the rules and categories I’ve laid out above. But due to the highly structured nature of music, it lends itself to being named and categorized.

I hope you’ve found this blog post useful. As a pretty analytical person and a lover of language, I really enjoy having terms to use to refer to the things I’m thinking off, and have found the terminology defined here very useful.

I also have no way to know for sure, but I strongly suspect that knowing these things has helped me become a better dancer. It helps me remember and make sense of things I’ve been taught in lessons. And when I make a mistake in my musicality, it also helps me make sense of what went wrong it later so that I can improve!

How to Type Non-English Characters on an English Keyboard (in Windows)

Occasionally for one reason or another, someone asks me how I type special characters such as accent marks in a “normal” keyboard (ex: Me gustaría ir a Cancún este año.) Leaving aside that U.S. keyboards are no more or less normal than any other country’s keyboards, it’s a pretty good question without an obvious answer. There are two main ways I use to type these sorts of characters:

Alt Codes

This method works on any Windows computer with a standard keyboard and requires no setup. Simply hold down the alt key and then type numbers on the number pad (if you don’t have a number pad on your keyboard, check out this article for instructions.) Example: alt+164 creates the ñ character.

Seriously, you can try it right now! Open Notepad and start holding alt and then typing 2-3 digits on the numpad! You can create all sorts of interesting characters this way. ♥♪♫♀☼§☺⌂

Unfortunately this requires memorizing the appropriate alt codes for the characters you want to type, but many languages only use 5 accented vowels plus a handful of other characters, so learning them isn’t too onerous. You can use online resources like those at altcodeunicode.com to find the ones you want for a given language.

United States – International Keyboard

This requires you to customize the language settings for your operating system, but is my favorite method of writing non-English characters for its ease and versatility.

To get started, open your “Region & Language Settings” and select the options for English (United States):

ApplicationFrameHost_2019-03-10_16-50-51.jpg

Then Add a keyboard:

ApplicationFrameHost_2019-03-10_16-52-24.jpg

Then select the United States-International keyboard:

ApplicationFrameHost_2019-03-10_16-53-06.jpg

Now Windows recognizes two different keyboard layouts. You can switch between them by clicking on the new keyboard selector which will appear on your taskbar when you have multiple keyboard layouts, or by pressing Windows Key + Spacebar.

2019-03-10_16-54-12.jpg

Once you have United States-International selected, now you can type right-leaning accent marks my typing an apostrophe followed by any vowel. As an example: ‘ + a (Ex: á.) You can combine all sorts of punctuation with characters this way:

  • Backticks (`) become left-leaning accent marks: à
  • Apostrophes (‘) become right-leaning accent marks: á
  • Quotes (“) become umlauts: ä
  • Tildes (~) combine with many letters like n: ñ
  • Carets also combine with vowels: â

And if you want to type one of these characters by itself, either follow it with a character that it can’t combine with, or simply hit spacebar if you’re not sure. Overall I find that this keyboard layout gives me a pretty straightforward way to type characters in most of the other languages I’ve learned, while still being able to type in English as I normally would.

For more information, check out Microsoft’s Support article on the United States International Keyboard.


Hopefully if you have found yourself attempting to write in another language, but stumped by how to get your keyboard to type the right characters, this will help!

My Favorite Programming Blogs

I maintain a list of programming blogs which have had made a big impression on me, so that I can go back and read them once in a while. I recently realized that there’s probably people who would also like to go through them, so… here they are!

No Deadlines For You! Software Dev Without Estimates, Specs, or Other Lies

This is one of my favorites, because it deals directly with connecting dev work with business value – a gap that is both very important, and very challenging to bridge:

The core idea is: put uncertainty and risk at the center of a conversation between the developers and the rest of the business (instead of everyone pretending such nasty things don’t exist). Doing so allows the entire business to tackle those genuine challenges together.

6 Reasons Users Hate Your New Feature

The truth is that users will often ask you for a solution when it would really be more helpful to tell you that they have a problem. […] Sometimes users will tell you that they want a toaster in their car, when what they really mean is that they don’t have time to make breakfast in the morning.

Blameless PostMortems and a Just Culture

We need to avoid [the cycle of blame]. We want the engineer who has made an error give details about why (either explicitly or implicitly) he or she did what they did; why the action made sense to them at the time. This is paramount to understanding the pathology of the failure. The action made sense to the person at the time they took it, because if it hadn’t made sense to them at the time, they wouldn’t have taken the action in the first place.

What the Heck is a Relation? From Tables to Cartesian Products to Logic

This is a great blog about relations from a mathematical perspective, explained very well in layman’s terms, which I found really helpful in creating a mental model for why relational databases are so darned good at modeling real world things.

Say NO to Venn Diagrams When Explaining JOINs

It wasn’t until I first read this article that I realized why the Venn Diagrams for JOINs are hard to remember: because they’re wrong. Rather, this article explores using JOIN diagrams to explain JOINs, which is a much better way to think about them.

The Little Mocker

A great blog by Bob Martin on various types of test dummies, doubles, and mocks. A good read if you’re into TDD.

Why Most Unit Testing is Waste

While I’m a big fan of automated testing and TDD, it’s important to understand the counterarguments. This blog makes a great case for balancing the cost of creating, running, and maintaining a test suite against the amount of risk it helps you mitigate.

So when they wrote their first function for this project three years ago they wrote a unit test for it. The test has never failed The question is: How much information is in that test? That is, if “1” is the passing of a test and “0” is the failing of a test, how much information is in this string of test results:

11111111111111111111111111111111

There are several possible answers depending on which formalism you apply, but most of the answers are wrong. The naive answer is 32, but that is the bits of data, not of information. You could be an information theorist and say that the number of bits of information in a homogeneous binary string is the binary log of the length of the string, which in this case is 5. But that isn’t what I want to know: in the end I want to know how much information I get from a single run of this test. Information is based on probability. If the probability of the test passing is 100%, then there is no information — by definition, from information theory. There is almost no information in any of the 1s in the above string. (If the string were infinitely long
then there would be exactly zero bits of information in each test run.)

The Codeless Code

A collection of fictional stories – each of which is a metaphor for a programming lesson or problem, written in the spirit of Zen kōans. Some of my favorite entries are The Tool-Shed and The Hidden Variable.

Never Heard Of It

An article about learning to be honest about not knowing things, and in a profession characterized by constant change and learning, how to know which things are worth knowing.

As I surveyed the patterns in my daily information bombardment, one dichotomy appeared rather quickly. Boiling it down to a quick litmus test: some things can be easily Googled for when needed, and some things cannot. This is a useful barometer, a differentiator between things to reference versus concepts to know.

Programmer, Interrupted

“A programmer loses X minutes of productivity when you interrupt them” is a common phrase that is bandied around in the programming industry – universally acknowledged, but tough to communicate to non-programmers and rarely backed up with research. This blog does exactly that: explains some research that demonstrates this to be true.

Joel on Software

Joel Spolsky was a very well-known blogger on programming and software design topics before co-creating the little-known website Stack Overflow. I thought about picking one article, but honestly the blog is filled with many gems and in my opinion any career programmer should read through it at least once. Even when I disagree with him, he explains his point well and in an entertaining way. Some highlights:

Programming Sucks

A humorous article that makes fun of the programming industry with a mix of metaphor and hyperbole.

“Double you tee eff?” you say, and start hunting for the problem. You discover that one day, some idiot decided that since another idiot decided that 1/0 should equal infinity, they could just use that as a shorthand for “Infinity” when simplifying their code. Then a non-idiot rightly decided that this was idiotic, which is what the original idiot should have decided, but since he didn’t, the non-idiot decided to be a dick and make this a failing error in his new compiler. Then he decided he wasn’t going to tell anyone that this was an error, because he’s a dick, and now all your snowflakes are urine and you can’t even find the cat.

The Wisdom of James Mickens

Also known as “the funniest man at Microsoft Research” – his articles and presentations are unfailingly humorous, and he manages to mix his absurdist humor with pointed jabs at the state of the industry as a whole.

In some way that I don’t yet understand, I’m glad that theorists are investigating the equivalence between five-dimensional Turing machines and Edward Scissorhands. In most situations, GUI designers should not be forced to fight each other with tridents and nets as I yell “THERE ARE NO MODAL DIALOGS IN SPARTA.” I am like the Statue of Liberty: I accept everyone, even the wretched and the huddled and people who enjoy Haskell. But when things get tough, I need mission-critical people; I need a person who can wear night-vision goggles and descend from a helicopter on ropes and do classified things to protect my freedom while country music plays in the background. A systems person can do that. I can realistically give a kernel hacker a nickname like “Diamondback” or “Zeus Hammer.” In contrast, no one has ever said, “These semitransparent icons are really semi-transparent! IS THIS THE
WORK OF ZEUS HAMMER?”


More coming soon ™ – as a find new blogs or remember ones that I think should be included, I’ll add them here. Enjoy!

Navigating the Facebook App Review Process

Attempting to get a Facebook app approved for a new permission can be a very frustrating process. Despite my best efforts, I recently found myself waiting for 15 days and had to reapply no fewer than 6 times in order to get permission to link to users’ profiles from my Facebook app.

I found Facebook’s documentation – while voluminous – to be very disorganized which made it difficult to find exactly what I needed to move forward. Furthermore, the review process itself feels very opaque and bureaucratic, which was frustrating. But now that I’ve survived the process, and finally had my application approved, I wanted to offer some guidance to future developers attempting to get their App Reviewed.

Know What You Need

First and foremost, any permissions your app uses must be granted to your app when the user logs in initially. For a full list of permissions you can request during login, check out the Facebook Login Permissions documentation.

chrome_2019-02-25_22-18-01

The documentation gives guidance about valid use cases for each permission – so read up on the permission you hope to use and make sure your use case is allowed by Facebook before applying for App Review.

Also note that some permissions require Business Verification – this will not be available to Individual app developers and will require a Business profile to be associated with your app account, and you will have to sign a contract with Facebook attesting that you will not abuse your app permissions, and will comply with their information privacy rules etc.

Where to Ask for Help

Facebook has an official Facebook Developer Community group in which you can ask questions and gather feedback. This group is monitored by Facebook staff, and while I didn’t get an answer to every question I asked there, one of Facebook’s staff was able to help me past one very confusing roadblock that might have had me stumped for days otherwise. If you get stuck, consider asking a question in that group.

Double Check Your Business Use Setting

According to a Facebook staff member I spoke with in the group above, if you have your app configured for a business-to-business use case, then Business verification will be required for any permissions you request – even those permissions which would not normally require Business Verification.

Image may contain: text

This one caught me off guard since it wasn’t clear based on the Business Use options that individual developers not representing any business are meant to select “Support my own business” here. Selecting “Provide services to other businesses” indicates to Facebook that your target use-case is business-to-business which requires Business Verification.

Know About Test Apps

This one is crucial – the first time I applied for the user_link permission, I simply described how the permission would be used. There was no way for my app to request the permission, even from my Test Users, so I was surprised to have my application rejected saying that I must show a live demo of the feature – at first I really wasn’t sure how to even go about fulfilling this request.

It took me quite some time to find, and Facebook’s documentation doesn’t really point you toward it, but Test Apps are the solution to this issue. You can create a Test App associated with your app account. Test Apps can never be set to Live mode – meaning that real Facebook users will never be able to actually use them. But they can be used by your test users, and have access to every permission which allows you to use the App ID and Secret for your Test App to create a live demo of any permission you wish to request, and allow Facebook to test it in a live environment during app review.

Don’t Ask the Reviewer to Post Data to Your Website

Despite clear instructions to do so, I had my app rejected 4 times without much explanation when I asked the instructors to create a new post on my website using a test user which would have included the test user’s name and a link to their profile. You get very little feedback when your app is rejected – usually just “we couldn’t see how the permission is being used” with no further explanation.

Sometimes you’ll be lucky and they will include a screenshot. I’ve concluded that the reviewers are not allowed to submit data to the websites they are reviewing – which were key steps in my instructions to them. It seems that once they got to steps they weren’t allowed to do, they simply ignored my instructions and clicked around the site a bit, hoping to find the permission in use, and then gave me a cookie-cutter rejection notice, occasionally with a screenshot of the wrong page or logged in as the wrong user.

During the final review, I simply created an event on the website myself and shared a link to it, demonstrating that my user’s name linked to their Facebook profile, and my application was approved upon verifying that the link worked.

Be Explicit About Which Test App/User the Reviewer Should Use in Your Instructions

In one of my rejections, I was given a screenshot of the error page that a user sees when attempting to follow a profile link issued to a Test App when logged in as a real Facebook user. This indicates that the reviewer was testing with a real Facebook account, and that they had ignored the selection in the App Verification pane where you can provide a Test App name.

This may be in part because the UI of that page is broken – the Test User field autocompletes with test user names, but only from your main app, and not the selected Test App. Since Test Users of your live app can’t use your Test App, I’m not sure exactly how this was meant to be used. But in my application which was accepted, I simply indicated which Test App and Test User should be used in the step-by-step instructions I provided, and that seems to have done the trick.

Conclusion

I hope that these tips help ease the experience of anyone else attempting to get a Facebook App Reviewed for a new permission. Ultimately the experience was extremely frustrating for me and ended up spanning more than two weeks from when I first applied to when my app was approved. But hopefully by following these tips and reading the linked documentation, things will go smoother for you!

What to Log in .NET

In my previous post, I discussed a lot of the when of logging – when to log things in your application. So in this post I’d like to talk a little bit about what to log.

Having lots of logs is certainly better than having none, but if the logs don’t capture important information that allow you to make decisions about your application design, then they are not as valuable as they could be. So in this post I will discuss certain key pieces of information which can greatly enrich your logs:

Web Request Data

In a web application, it can be hugely useful to log certain information about the incoming request. The first things we want to take a look at are what was requested. See below for WebForms/MVC code snippets:

Protocol

If your web server serves both HTTP and HTTPS requests, you should log the protocol being used for the request. (You do use HTTPS, right?)

Hostname

By logging the hostname, we can see the actual domain that was requested, including ports (if the port was specified) and the subdomain.

Path

This represents the actual request path – the part after the hostname and before the query string. In MVC apps, this is used for routing to the appropriate controller/view, and in WebForms apps this will be the path to the .aspx page which is rendered.

Query String

Because query strings are used for passing in important information, the request information is not really completed without it. NOTE: if you are passing sensitive data via query strings, be sure not to log them! Additionally, see this StackExchange post for reasons why you should not do this.

IP Address

This one is a common pitfall, but it’s good information to have no matter what. Remember that the IP address is the request of the machine which sent the request to you, and not unique to a user. For instance, if multiple people in the same home visit your website on different computers, you will most likely see the same IP address for all of them.

To add to the complexity, if a proxy, CDN, or load balancer forwarded the request to you on behalf of a user, then this IP address may not be the user’s at all. But take heart: well-behaved proxies will usually include the original IP in a header which you can retrieve. A common way of doing this is with the X-Forwarded-For header. Some other services such as Cloudflare and Akamai instead use the True-Client-IP header. Depending on your network infrastructure will determine where the actual user’s IP address will be stored in the request.

User Agent

The User Agent is another important header to capture – it indicates the web browser and operating system that the user is using. This can be very useful when attempting to diagnose bugs that only occur on certain browsers, and can also give insight into the technologies in use by your users. At work we were shocked to find that an extremely small percentage of our users were viewing our website on Android TVs!

Code Example: WebForms and .NET Framework MVC

Below is an example of how to fetch all of these values in an MVC or WebForms app. In your Controller/Page, there is a Request property defined which contains lots of information about the request:

var hostname = Request.UserHostName;
var path = Request.Path;
var queryString = Request.QueryString;
var ip = Request.UserHostAddress;
var forwardedFor = Request.Headers["X-Forwarded-For"];
var trueClientIp = Request.Headers["True-Client-IP"];
var userAgent = Request.UserAgent;

Code Example: .NET Core MVC

The implementation of the MVC Controller’s Request property’s class is a little different in .NET Core, but the same principles still apply here:

var hostname = Request.Host;
var path = Request.Path;
var queryString = Request.QueryString;
var ip = Request.HttpContext.Connection.RemoteIpAddress;
var forwardedFor = Request.Headers["X-Forwarded-For"];
var trueClientIp = Request.Headers["True-Client-IP"];
var userAgent = Request.Headers["User-Agent"];

Code Example: ASP.NET Core Owin

If you need to retrieve the request data from the Owin context during the execution of your Owin pipeline (this is a great place for performance and error logging!) then it is very similar to the above example, except that we will get the request object from the Owin context instead of a property on our Controller:

app.Use(async (context, next) =>
{
  var hostname = context.Request.Host;
  var path = context.Request.Path;
  var queryString = context.Request.QueryString;
  var ip = context.Request.HttpContext.Connection.RemoteIpAddress;
  var forwardedFor = context.Request.Headers["X-Forwarded-For"];
  var trueClientIp = context.Request.Headers["True-Client-IP"];
  var userAgent = context.Request.Headers["User-Agent"];
  return await next();
}

User Data

It is very useful to have information about the user themselves when looking at logs. This lets you know who caused the log, but can also be a useful aid when troubleshooting bugs, since you can create a test account with similar data. It is also very useful for audit logging, if you need to answer a question about which user took an action and when.

To get data about the current user, you will need access to the user’s Principal. How you get access to this will vary by environment:

  • In WebForms or .NET Framework MVC apps, the Page/Controller has a User property which will contain the current user’s Principal.
  • In most .NET Framework apps, you can also access the current user’s Principal using Thread.CurrentPrincipal.
  • In .NET Core apps, Thread.Current principal is not available, and you should instead use .NET Core’s dependency injection and the IHttpContextAccessor interface.
  • In the Owin pipeline, you can access the User property of the HttpContext, which contains a ClaimsPrincipal.

Once you have access to the user’s principal, you can use it to check whether they are properly authenticated, and get their name:

var userPrincipal = Thread.CurrentPrincipal;
var isAuthenticated = userPrincipal.Identity.IsAuthenticated;
var userName = userPrincipal.Identity.Name;

Additionally, a very common implementation of the IPrincipal interface is the ClaimsPrincipal, which can contain all sorts of information about the current user – common and useful claims to look at are the user’s roles, email, and any application-specific claims you might generate during user login, such as a user ID or customer ID.

Caller Data

A very useful diagnostic tool that I’ve come to love is: including the file and method which actually wrote the log. While you can rely on stack traces in the case of exception logging, it can also be useful to find out where other types of logs, such as user behavior logs are being generated, in order to put them into context.

C# exposes a couple of really handy attributes for this purpose which can be applied to your method parameters. The first is the CallerMemberNameAttribute which contains the method name of the method which called yours. The second is the CallerFilePathAttribute, which contains the path on disk to the source file in which the calling class is defined (note that the path will usually correspond to the machine which compiled the code and not necessarily the one on which it is running!)

To use these attributes, you simply need to add them to nullable string parameters on your method which does the logging:

private void CallerExample([CallerMemberName] string memberName = null,
    [CallerFilePath] string filePath = null)
{
  var fileName = filePath.Substring(filePath.LastIndexOf(Path.DirectorySeparatorChar) + 1);
}

Bear in mind that the values at runtime will be the calling method – even if you use some private helper methods to do logging, in which case the method and file name may always point to your logging class. In this case, simply add the parameters with the Caller attributes to your public method and pass the values in to your private method – the values you pass in as arguments will be used instead of the actual caller values.

Environment Data

Capturing information about the environment your code is running in can be very useful when diagnosing environmental issues, such as when an error only occurs on one server for some reason.

Some of the most useful pieces of information here are:

  • Machine Name (useful to know if something is only happening on one machine.)
  • Thread ID (useful for diagnosing race conditions.)
  • UserAccount under which the application is running (useful for diagnosing permissions issues.)

These values can be accessed on the Environment static class:

var machineName = Environment.MachineName;
var threadId = Environment.CurrentManagedThreadId;
var userAccount = Environment.UserName;

Error Data

This one is fairly easy, but is nonetheless very important. Whenever your application detects an error or exception, you should log the Exception message and stack trace for diagnostics.

Many programmers simply log the exception itself and think that they’re done, but remember – your exception logs will be the first place you look when diagnosing a bug! Think ahead and put yourself in your shoes and consider what additional information might help you get to the bottom of a tricky bug, and log it now! Good candidates for this are things like method parameters, collection sizes, or user roles if an authorization exception is possible.

Conclusion

I hope that this has given you some useful ideas and code examples to really up your logging game. As you enrich your logging story by adding additional data to logs, and a more thorough system of logging to your application, you will find yourself learning about all sorts of things that happen during your program’s execution that you may never have discovered through normal testing.

Techniques and Advice for Logging in .NET

When setting out to design maintainable software, one of the most critical things to bear in mind is creating feedback loops that allow you to make code decisions based on the application’s actual use and performance under real-world circumstances. A crucial aspect of this is that your application needs to be able to tell you when something is wrong.

One of the main ways to achieve this is by application logging. There are many forms of logging, but in the scope of this article we will discuss a few categories of logging:

Diagnostic Logging

Diagnostic logging is, broadly logs that help you identify and troubleshoot failure scenarios in your application. Below we will discuss a variety of ways to employ diagnostic logging to help reduce your time spent tracking down tricky bugs to help you spend more time on the fun parts of programming.

Catchall Exception Logging

This is the most basic form of exception logging, but its great benefit is that if any code is written in your application that does not correctly handle exceptions, you are guaranteed to have a log about it! Depending on the structure of your application this may take many forms.

In ASP.NET applications, you can define a method in your Global.asax file to handle all errors that occur during processing of an HTTP request:

void Application_Error(object sender, EventArgs e)

In an application using the Owin pipeline, you can write a simple middleware which wraps all subsequent middleware executions in a try/catch, something like this:

app.Use((context, next) =>
{
  try
  {
    return next();
  }catch(Exception ex)
  {
    // Log the exception here.
  }
}

In .NET desktop applications, you can use the AppDomain class’ UnhandledException and FirstChanceException lifecycle events to log either all exceptions or only unhandled exceptions, whichever you prefer (note that in recent versions of .NET Core console apps this works as well!):

AppDomain.CurrentDomain.UnhandledException += (sender, args) =>
{
  var ex = args.ExceptionObject;
  // TODO: test the exception object type and log it
};

Downstream Dependency Error Logging

Any time your application interacts with a downstream dependency, such as a database or another API, you should wrap the interaction in a try/catch and log any exceptions that get thrown. This is incredibly useful for determining when failures in your application are caused by bugs or outages in other people’s code!

try
{
  myDownstreamDependency.SendRequest()
}catch(Exception ex)
{
  // Log the exception here.
}

Note that when doing this, you should also include as many specifics about what you were doing in your log. Things like: which API call you were making, what URL you were accessing, what arguments you passed in, which user was taking the action, etc.

Soft Failure Logging

As we are adding logging to our application, it is important to remember that not all failures in our code result in exceptions being thrown. There are other code failures which can result in bad/missing data without ever throwing an exception. One good example of this is when you know an operation should always return a non-empty list, but the resulting list is empty. While this may not cause an exception that causes your application to be unusable, it is still a logical error which will have some known or unknown impact on your users and should be logged.

Performance Logging

Another thing we want to know about in our application is when something causes it to run slowly. In many cases, problems or changes in our code can cause the code to begin running more slowly, which from the user’s perspective is a bug!

So any operation which a user initiates which could cause a bad user experience if it were to take too long should be wrapped in some form of performance logging. A great tool for this is the .NET Stopwatch class.

var stopwatch = new Stopwatch();
stopwatch.Start();
// Do something
stopwatch.Stop();

After calling the Stop method on our Stopwatch, then we can check its ElapsedMilliseconds or ElapsedTicks properties to check how much time has passed, and include this information in our logs.

HTTP Request Performance Logging

Logging the performance of all HTTP Requests is a great way to get a high-level overview of the performance of a web application. Depending on your architecture there are a few ways of doing this, but in ASP.NET applications with a Global.asax file, you can define methods to handle the ASP.NET lifecycle events for the beginning and end of handling a request:

private Stopwatch _stopwatch;
protected void Application_BeginRequest()
{
  _stopwatch = new Stopwatch();
  _stopwatch.Start();
}
protected void Application_EndRequest()
{
  _stopwatch.Stop();
  // TODO: log the performance here
}

In an Owin application (such as ASP.NET Core apps), it’s even easier, you can just define a middleware near the beginning of your Owin pipeline that calls the rest of your middleware, and log the performance there:

app.Use((context, next) =>
{
  var stopwatch = new Stopwatch();
  stopwatch.Start();
  var result = next(); // We run all of our other middleware
  stopwatch.Stop();
  // Log the performance here.
  return result;
}

Downstream Dependency Performance Logging

Once again it is very important to keep track of your downstream dependencies. Not all problems in your application originate in your code, so being able to tell when slowness in your application originates in an API or database you depend on is very important to point your troubleshooting in the right direction (and point blame in the right direction, if you have the misfortune to work for an organization which does not have blameless postmortems…)

As with our downstream dependency error logging, we should capture as many specifics as possible about our API call/DB query in order to facilitate rapid troubleshooting of the problem.

User Behavior Logging

It can be very useful to know how your users are actually using your application. The scope of this extends beyond just troubleshooting bugs and problems, to informing your technical priorities and even feature grooming. Below are some suggestions about useful types of user actions to log:

Log In/Out

Whenever a user begins or ends using your application, you should log that behavior. This provides very insightful usage statistics. Additionally, when a user attempts to log in, you should record whether their login attempt succeeded or failed – this is useful for detecting brute force and DDoS attacks, or even informing your product decisions around things like password recovery or failed password lockout thresholds/durations.

New Features

When you release a new feature, it can be very useful to log how many users are using the feature and how they are using it. This is important feedback for an Agile development process – you can’t always talk to your users about how they are using your application, but you can log what they do! Combining this with error and performance logs can give you a lot of insight into whether users are having a positive experience with a new feature.

High-Impact Features

If a particular feature in your application is known to have a significant performance impact, it can be very good to log the frequency with which users use that feature. This might be operations such as provisioning a new virtual machine, or clearing all of the cached data for their customer.

Code Behavior Logging

Just as User Behavior Logging is useful for finding out what your users are doing, Code Behavior Logging is useful for finding out what your code is doing.

Application Lifecycle

Logging key events in your application lifecycle can be really useful for diagnosing startup problems and crashes. In particular, one of the first things you do when your app starts is create a log with the current machine name! This not only gives you a history of when your app was started or restarted, but also lets you make sure that your logging is working!

In a .NET Core application, the easiest place to do this is right in the beginning of the Main method:

public static void Main(string[] args)
{
  // Write your log here!
  CreateWebHostBuilder(args).Build().Run();
}

For older .NET Framework WebApps (MVC/WebForms), you will instead need to do this during the Application_Start HttpApplication lifecycle event in your Global.asax file:

protected void Application_Start()
{
  // Write your log here!
  // Usually startup things like dependency injection etc. go here.
}

 

For WPF desktop applications, the App.xaml.cs constructor or the MainWindow’s constructor are good places.

Other useful lifecycle events to capture are: application shutdown, and the execution of any recurring background worker threads which may run in the background during normal operations.

Branch Execution

This is particularly useful in legacy codebases. Sometimes you will find yourself staring at some old code and wondering “does this code even ever get hit any more?” Branch execution logs can answer this question for you – simply add a log statement in that code branch and release it to users. It won’t take long before you’ll have an answer to that question.

Data Logging

When it is not clear what the actual data in your application looks like at runtime, it can be useful to log the data itself or key attributes about it. For instance, if you are doing lots of string operations in your application, it might be useful to log the length of those strings so that you know how important optimizing your string operations is. If the strings are always small, the optimization work may not be valuable enough to do. But for long strings it might warrant a higher priority.

Audit Logging

Audit Logging is designed to give you a record of sensitive user behaviors. There are many concerns involved in audit logging – audit logs are often customer-facing and can sometimes play a role in compliance and legal matters. So it’s really important to get these ones right.

User-Facing Data Changes

Any time a user modifies data that users have access to, and that change is persisted anywhere (such as a database or filesystem) you should generate an audit log for this behavior. Be sure to log not only the data that was modified, but the user doing it, and if possible, some details about what was changed.

Viewing Sensitive Data

While we are normally worried about auditing when users change data, some data is so sensitive that even viewing it warrants an audit log entry. What data falls into this category will vary depending on your application, but good candidates are financial information, or when one user views personal data about other users.

Conclusion

I hope that this blog has given you some good heuristics to think about when adding logging to your application. Logs are an absolutely amazing tool when it comes time to troubleshoot a bug, diagnose an outage, or provide customers with really specific and useful data about a data loss scenario. When things go wrong (and they will) logs are what helps you put on your Sherlock Holmes hat and make sure it doesn’t go wrong twice!

The Four Phases of Dance DJs

In my experience, as DJs gain more experience, they go through several phases. I’d like to discuss them today, to help my friends who are just starting out DJing dances think about how to level up their DJ game as quickly as possible:

Phase 1: The “Music I Like” DJ

This is where most DJs begin their journey: they have created a nice little collection of music they love to dance to, and they create playlists from that. It’s all of their favorite songs, and they’re able to put together 1-3 hours sets containing only songs they enjoy.

This is a great start, and the community can always use eager people willing to volunteer their time to DJ for us! But the fundamental problem with this style of DJing is that it fails to account for other people’s tastes. By creating playlists that appeal perfectly to one person’s preferences, you may accidentally also be creating playlists that don’t appeal at all to other dancers who come to your dance!

Eventually most DJs notice this and begin to move into Phase 2.

Phase 2: The “Music the Dancers Like” DJ

These DJs have expanded their collection to include the most popular songs for their dance style – even the ones they personally don’t like. Their playlists still contain some of their absolute favorites and some guilty pleasures, of course – but the majority of their set is designed to appeal to their audience rather than their personal tastes.

This is a huge improvement from the first phase of DJ’ing because by taking other people’s tastes into account, you will rarely have your dancers going home thinking “man, that DJ didn’t play a song I liked all night!” But DJs in this phase often give little thought to the progression of their music throughout the night – whether their transitions between songs makes sense, and whether they are playing so many similar songs in a row that their dancers who would prefer something different are getting bored.

As DJs endeavor to bump their game up to the next level, they move on to Phase 3.

Phase 3: The “Perfect Playlist” DJ

These DJs invest a lot of time, love, and energy into their playlists. They pick the first song carefully, making sure that their first song is a popular one that kicks off their set with a bang and gets everyone on the floor. They pay close attention to transitions between songs, making sure not to have too many songs of the same tempo/style in a row, to give their dancers a breather between fast songs, and making sure that the vibe doesn’t become sleepy with too many slow songs in a row.

chrome_2019-01-13_23-35-32.jpg

They also make sure that their dancers with particular tastes don’t have to go too long between songs that they really love – trying to mix in various styles at regular intervals. This gives their dancers a nice rhythm for the evening, taking social breaks on the songs they like less, but knowing it will only be 5-10 minutes until a song they will love is played. Everyone needs to take breaks sometimes, that’s expected. But we’ve all had the disappointing experience of going 15-20 minutes without hearing a song that really makes us want to jump up and dance.

But crowds change, tastes change, popular songs become played out, and we all know that the difference in vibe between late night dancing and early evening dancing is like night and day. So a great playlist, while it is a great start, still isn’t the top tier of DJing.

Phase 4: The “Handcrafted Playlist” DJ

These DJs create a playlist as a starting point, but know the old saying that “no playlist ever survives contact with the dancers.”

The Shire and everything after: Warband goes live!

Because every crowd, every event, every evening is different, these DJs update their playlists on the fly based on who showed up and what the energy level of the room is like. If they see a large group of people sitting down for a while, they keep changing songs until they manage to play something that gets them on the floor. As they do this, they learn the tastes of the various cliques of dancers while still feeling out and shaping the vibe of the evening. As the evening gets late, they play more slower songs, but also know that songs with a slow tempo (BPM) can nonetheless have very high energy.

They also know their community and play songs appropriate for the hour. We all know that as the evening goes on, the dance (and the crowd of dancers) evolves in predictable ways. Many West Coast Swing events accommodate junior dancers – this is great, but it means that songs with profane lyrics are better played at 2am than at 10pm. Likewise, they know that their older dancers will love to hear hit songs from decades past, and will plan to play those earlier in the evening when the crowd will best enjoy it.

Conclusion

As I’ve worked on my own DJing, I’ve managed to journey through the first 3 phases of DJing, and am now challenging myself to become a true Phase 4 DJ. If you are a DJ or an aspiring DJ, I hope that this article gives you some things to think about to find ways to up your own DJ game!

There’s nothing better than playing music for an energetic crowd all night, and then having people with varying tastes in music come up to you afterward and tell you they loved your set – that’s how we know we’ve really done our job well and created something special!