Present Like Nobody's Watching

Presenting online — more like a one-on-one discussion than a presentation in front of hundreds of people

Last week I presented my session High Performance SSRS at the inaugural GroupBy Conference. It was an incredibly fun experience getting to present a topic I'm excited about as well seeing others do the same at such a high caliber.

The thing that makes GroupBy different from other technical conferences is that it's free, it's community driven, and it's online-only. This makes it a very inclusive conference since barriers to attend are extremely low. The only significant barrier to watching the sessions live was a software restriction to the amount of live attendees. Fortunately, all of the sessions were made available on the GroupBy YouTube channel soon afterwards to allow anybody to watch the sessions at any time.

The 1,000 live attendee registration cap was quickly reached

Even though the conference is only half-over (more excellent presentations coming this Friday January 20 — go register!), my session is complete so I thought it would be fun to go over what made presenting at GroupBy different from an ordinary in-person conference.

Pre-Conference

There wasn't too much of a difference preparing for GroupBy compared to other speaking engagements. I still had to create a presentation that captured the audience's attention and told a good story. I also had a lot of rehearsing to do, especially since my presentation consisted of mostly live demos. In case you've never presented something with live demos, here's a secret: something always goes wrong. This means a lot of my preparation involved thinking about what could go wrong and trying to prepare for it so I wouldn't embarrass myself too badly in front of a live audience.

The main difference between this talk and others I've given is that I was entirely on my own in regards to audio and video setup.

Broadcasting from my kitchen on conference day

Having a both an audio (musician) and video (photographer) background, I probably went a little overboard with my setup. During my practice recording sessions, I kept finding little details that annoyed me and forced me to get to the setup that you see above. Ugly background? Move to the kitchen where there is a solid white wall. Bad lighting? Turn on all of the lights and bring in extra lamps. Laptop microphone sounds muffled? Pull out the condenser and boom stand.

Conference Day

Since I didn't have to travel anywhere to speak, I just went about my normal day and went in to work. We booked a conference room (zing!) and had a watch party, ate some pizza, and learned about SQL Server 2016 built-in performance boosts, automation with PowerShell, SQL at StackOverflow, SQL Server 2016 hidden gems, what not to do with SQL, and Power BI. During the lunch break, I went home to get ready for my session.

My actual session went well and was strangely comfortable to present since I got to do it in my home, in my regular programming chair, 3 feet away from my regular programming fridge (need caffeine from somewhere right?).

The one daunting aspect of my presentation was that it was difficult getting a sense of how the audience was following along. Although Brent was great at laughing at my jokes, commenting, and moderating audience questions, it was hard for me to judge if I needed to speed up sections or slow down and explain in more detail. With a live audience, I can read body language and adjust. Online, it felt like nobody is watching.

Post-Conference

People were watching however, because soon after my presentation was over I got flooded with texts, emails, and comments about my session. Session ratings flowed in over the next few days and I was happy to get feedback indicating that my session was well received.

And although there were no speaker dinners or sponsored after parties, Renee and I made up for it by going to our favorite sushi bar to celebrate!

Presenting in the virtual world was definitely a very different experience for me but I loved it. Would I do it again? No doubt, and I hope you do too (abstract deadline is February 28 — get typing!).

The Most Lacking Feature In SSRS (7 years and no fix?)

57615-0trztfz8w8a9bhszp

This post is a response to this month's T-SQL Tuesday prompt. T-SQL Tuesday was created by Adam Machanic and is a way for SQL users to share ideas about interesting topics. This month's topic is SQL Server Bugs & Enhancement Requests.


Being a frequent user of SQL Server Reporting Services for the past 6 years has made me blind to the annoyances that I had with the software when I first started using it. For example, I've learned to deal with not being able to easily change the order that reports appear in the Solution Explorer. I am no longer frustrated that I can't change the order of datasets in the Report Data pane. I've grown accustomed to editing the XML of the report to accomplish these annoyances and many others.

Even though I think SSRS is a superb piece of software (no other tool I know of can generate reports so easily with as much flexibility as SSRS) there is one lacking feature that still drives me nuts:

Why can't I organize my reports in sub folders!?

Don't be fooled by my photo manipulating skills, this New Folder button doesn't actually exist in SSRS.

Seriously. Visual Studio allows me to organize my files in sub folders in nearly all other project types: C# console apps? Check. ASP.NET MVC solutions? Check.

Why then can't my SSRS solutions do the same thing?

If you look at the Microsoft Connect for the issue more than 200 people agree. It's ridiculous this functionality isn't built in. Not only does Visual Studio have the capability in other solutions, but reports can be deployed to multiple different folders on the SSRS server itself, leaving the only missing link a context menu action that says "Create New Folder." I know what I'm asking for here is a "basic" change, nothing nearly as complicated as adding an additional QUALIFY filtering clause (which would be great to have too), but that just makes it more reason that this should have been fixed a long time ago!

However, there is a brief glimmer of hope. Microsoft has been releasing updates to SQL Server Data Tools somewhat regularly the past year, including bug fixes and feature requests from Connect feedback. Let's hope they continue to get better about fixing issues like these so that everyone will be able to right click > Create New Folder in their SSRS projects sometime in the near future.

Epic Life Quests

ff87c-0a7-0aztlscfuxve_

It is easy to get caught up in the daily details of life and not take the time to reflect on longer term goals and accomplishments.

Inspired by Brent Ozar and Steve Kamb, these Epic Life Quests are intended to help me reflect on my accomplishments and help me stay focused on the things that are important.

Each level contains five achievements and once all are completed I can "level up" to the next five. Follow along and let me know if you create any epic life quests of your own.

Level 2 (currently working on)

  • Blog weekly for 6 months straight — Last year I began blogging more than any previous year, but I didn't always stick to a schedule. My biggest problem was I didn't know what I wanted to write about so choosing topics was difficult and frustrating. After looking back at what posts were the most well-received, I've decided to focus the first half of 2017 to mostly technical and professional development type topics.
  • Speak at a technical conference — I've been presenting technical content in small user group type settings for years but this year I want to try my hand at some larger audiences. Completed January 2017 at the GroupBy conference.
  • Vacation in Hawaii — Our vacations in 2016 focused on places we could reach by car so that we could save some money for a larger trip. This will be the bigger trip. After visiting Hawaii, I will have visited 36 states + Washington D.C (airports don't count!).
  • Work on mental mindfulness— practice meditation to improve focus, patience, manage stress, be happier. I want to average at least 5 days/week for 3 months to reach this goal.
  • Always be reading at least one book — Although I read 40+ books in 2016, there were stretches of weeks at a time where I was not reading anything. For 6 months I don't want to go more than 3 days without having picked a book to have available to read.

Level 1 Quests (completed before 2017)

Here are some of my achievements before I started this page on January 1, 2017.

  • Set up an environment for programming regularly at home — completed 2016
  • Start using GitHubt for my coding projects — completed 2016
  • Built a hardware project that gets regular use (not just proof of concept) — completed 2016
  • Buy a house — completed 2015
  • Get into backpacking — completed 2015
  • Take a car trip in Europe with Renee — completed 2015
  • Learn to read books for pleasure — completed 2014

XmlReader vs XmlDocument Performance

306d1-15myowyb7a3ye9kbyzjdttg

Recently I have been working on a project where I needed to parse XML files that were between 5mb and 20mb in size. Performance was critical for the project, so I wanted to make sure that I would parse these files as quickly as possible.

The two C# classes that I know of for parsing XML are XmlReader and XmlDocument. Based on my understanding of the two classes, XmlReader should perform faster in my scenario because it reads through an XML document only once, never storing more than the current node in memory. On the contrary, XmlDocument stores the whole XML file in memory which has some performance overhead.

Not knowing for certain which method I should use, I decided to write a quick performance test to measure the actual results of these two classes.

The Data

In my project, I knew what data I needed to extract from the XML up front so I decided to configure test in a way that mimics that requirement. If my project required me to run recursive logic in the XML document, needing a piece of information further down in the XML in order to know what pieces of information to pull earlier on from the XML, I would have set up an entirely different test.

For my test, I decided to use the Photography Stack Exchange user data dump as our sample file since it mimics the structure and file size of one my actual project's data. The Stack Exchange data dumps are great sample data sets because they involve real-world data and are released under a Creative Commons license.

The Test

The C# code for my test can be found in its entirety on GitHub.

In my test I created two methods to extract the same exact data from the XML; one of the methods used XmlReader and the other XmlDocument.

The first test uses XmlReader. The XmlReader object only stores a single node in memory at a time, so in order to read through the whole document we need to usewhile(reader.Read()) in order to loop all of the nodes. Inside of the loop, we check if each node is an element that we are looking for and if so then parse out the necessary data:

public static void XmlReaderTest(string filePath)
{
    // We create storage for ids of all of the rows from users where reputation == 1
    List<string> singleRepRowIds = new List<string>();

    using (XmlReader reader = XmlReader.Create(filePath))
    {
        while (reader.Read())
        {
            if (reader.IsStartElement())
            {
                if (reader.Name == "row" && reader.GetAttribute("Reputation") == "1")
                {
                    singleRepRowIds.Add(reader.GetAttribute("Id"));
                }
            }
        }
    }
}

On the other hand, the code for XmlDocument is much simpler: we load the whole XML file into memory and then write a LINQ query to find the elements of interest:

public static void XmlDocumentTest(string filePath)
{
    List<string> singleRepRowIds = new List<string>();

    XmlDocument doc = new XmlDocument();
    doc.Load(filePath);

    singleRepRowIds = doc.GetElementsByTagName("row").Cast<XmlNode>().Where(x => x.Attributes["Reputation"].InnerText == "1").Select(x => x.Attributes["Id"].InnerText).ToList();
}

After writing these two methods and confirming that they are returning the same exact results it was time to pit them against each other. I wrote a method to run each of my two XML parsing methods above 50 times and to take the average elapsed run time of each to eliminate any outlier data:

public static double RunPerformanceTest(string filePath, Action<string> performanceTestMethod)
{
    Stopwatch sw = new Stopwatch();

    int iterations = 50;
    double elapsedMilliseconds = 0;

    // Run the method 50 times to rule out any bias.
    for (var i = 0; i < iterations; i++)
    {
        sw.Restart();
        performanceTestMethod(filePath);
        sw.Stop();

        elapsedMilliseconds += sw.ElapsedMilliseconds;
    }

    // Calculate the average elapsed seconds per run
    double avergeSeconds = (elapsedMilliseconds / iterations) / 1000.0;

    return avergeSeconds;
}

Results and Conclusions

Cutting to the chase, XmlReader performed faster in my test:

Performance test results.

Now, is this ~.14 seconds of speed difference significant? In my case, it is, because I will be parsing many more elements and many more files dozens of times a day. After doing the math, I estimate I will save 45–60 seconds of parsing time for each set of XML files, which is huge in an almost-real-time system.

Would I have come to the same conclusion if blazing fast speed was not one of my requirements? No, I would probably go the XmlDocument route because the code is much cleaner and therefore easier to maintain.

And if my XML files were 50mb, 500mb, or 5gb in size? I would probably still use XmlReader at that point because trying to store 5gb of data in memory will not be pretty.

What about a scenario where I need to go backwards in my XML document — this might be a case where I would use XmlDocument because it is more convenient to go backwards and forwards with that class. However, a hybrid approach might be my best option if the data allows it: if I can use XmlReader to get through the bulk of my content quickly and then load just certain child trees of elements into XmlDocument for easier backwards/forwards traversal, then that would seem like an ideal scenario.

In short, XmlReader was faster than XmlDocumet for me in my scenario. The only way I could come to this conclusion though was by running some real world tests and measuring the performance data.

So should you use XmlReader or XmlDocument in your next project? The answer is it depends.

Coffee's Variety Problem

Freshly roasted coffee beans from central Mexico.

Spaghetti Sauce Origins

During the 1970s consumers had a limited number of spaghetti sauces that they could purchase at their local supermarket. Each store-bought sauce tasted the same, developed in test kitchens from the average flavor preferences of focus groups.

During this time, Prego brand spaghetti sauce was new to the market and was having a difficult time competing against the more established spaghetti sauce brands like Ragu. Prego tasted the same as all of the other brands so capturing loyal customers from other brands was a challenge.

Struggling to become competitive, Prego brought on board Howard Moskowitz. Today some consider Moskowitz the godfather of food-testing and market research, but during the 1970s Moskowitz's beliefs on product-design were contrary to the mainstream establishment. Moskowitz believed that using the average preferences of focus groups to develop a product lead to an average tasting spaghetti sauce: acceptable by most, loved by none.

Instead of giving focus groups similar tasting sauces, Moskowitz decided to test the extremes: have groups compare smooth pureed sauces with sauces that have chunks of vegetables in them, compare mild tasting sauces with ones with lots of spice, and so forth.

The results of this type of testing may seem obvious today, but at the time they were unheard of: people prefer different kinds of spaghetti sauce. It was better for a food company to have a portfolio of niche flavors rather than try to make one product that appealed to everybody.

Today spaghetti sauce no longer has a variety problem.

After all of this initial research Prego made Extra-Chunky spaghetti sauce, which instantly became a hit with all of the people who prefer a thicker type of tomato sauce. In the years since, spaghetti sauce manufactures have caught on to the technique and supermarket shelves today are lined with regular, thick and chunky, three-cheese, garlic, and many other types of sauces that appeal to a wide-array of consumer flavor preferences.

Coffee's Variety Problem

Coffee today has the same problem that spaghetti sauce did a quarter-century ago.

The average supermarket's coffee aisle may look like it has a wide variety of choices: whole beans versus ground coffee versus coffee pods, arabica versus robusta beans, caffeinated versus decaf, medium versus dark roast, etc…

However, once you take a look at what coffee a particular individual may drink — let's say whole bean, medium-roast caffeinated arabica beans -the amount of variety found at a super market is surprisingly small, maybe only 2 or 3 different types.

The limited selection for whole bean, medium roasted coffee at my local super market.

Additionally, it is nearly impossible to know how long the coffee has been sitting unsold on the shelves, it's difficult to find a variety of coffee from different geographic regions around the world, and only if you are extremely lucky can you find a coffee that has been lightly roasted.

Independent coffee shops, especially those that roast their own coffee, offer slightly better options in terms of variety, however they come at a steep price: it is not unusual to see these coffees selling for \$16-\$28 per pound.

I understand these small-batch roasters experience higher costs due to lack of scale, availability of beans, and a slowly developing market to third-wave premium coffee, however paying \$20 for a pound of coffee beans (or even worse, the standard 12oz bag) is not something I can justify doing regularly.

This is what caused me to set out on my quest to get premium quality coffee beans at an affordable price.

The Internet Cafe

While the internet offers advantages in buying premium roasted coffee beans, there are still issues with unknown freshness and high prices, especially when the cost of shipping is included.

Shipping costs and price per pound can be reduced when buying in bulk, but buying in bulk means I'll have roasted beans sitting around for a long time before being consumed, therefore affecting freshness and taste. Short of finding friends who want to split a large coffee order, buying roasted coffee beans online isn't a great option.

What is a great option is buying green, or un-roasted, coffee beans. The shelf life of green beans is up to one year if stored properly and green beans are significantly cheaper than roasted beans because there is less processing involved.

Green, un-roasted coffee beans.

Not only are green beans fresher and cheaper, there is significantly more variety available. Commercial roasters need to buy beans in large quantities in order to be able to sell to coffee shops and supermarkets. This means they are sourcing coffee beans from large commercial farms that are able to supply such a large amount of coffee beans.

If we buy green beans for personal consumption at home, the number of farms that we can buy from is hugely expanded since we only need to buy in small quantities. A farm that only produces a few hundred pounds of beans each year is now within our grasp since commercial roasters would never be able to purchase from them.

There are many retailers online that cater to the home roasting market. My favorite is http://sweetmarias.com and they always have a huge selection of premium beans from all over the world, many for under \$6/pound.

Home Roasting

Roasting beans at home used to be the norm in the early nineteenth century. Roasting coffee beans is similar to making popcorn and can be done over a fire, in a stove, or in the oven. While these methods work, they are messy and involved. Fortunately, cleaner and more scientific options exist.

Commercially available home roasters are one such option, however they cost several hundred to thousands of dollars — well out of my price range.

The most easily obtainable and easy to use home coffee roaster is an electric air-powered popcorn popper. These retail for between \$15–\$25 new. If you decide to try this route, get one without a mesh screen on the bottom.

Once you have an air popcorn popper, roasting coffee beans is as easy as pouring them in, turning on the heat, and waiting until they turn the brown color you are accustomed to seeing.

Although roasting beans can be as easy as turning on the popcorn popper and waiting until the beans reach a desired color, there is a scientific rabbit hole that roasting geeks like me eventually wander down…

Home Roasting 2.0: Web Roast

When I first started roasting coffee beans at home, I started with the air popper method. I soon wanted to start experimenting more with how to make the process more automated, as well as how I could become more precise and play with different variables in order to change the flavors of my roasted coffee.

There are some mods you can make to a regular air popcorn popper to give you more control in how your roast your beans, but ultimately I wanted more control.

I present to you, Web Roast:

The home-made, Internet of Things (IoT) coffee roaster.

The finished project with all code can be found on my project's GitHub page: https://github.com/bertwagner/Coffee-Roaster/

Essentially, this is still an air popper but with much more control. The key features include:

  • Air temperature probe
  • Ability to switch the fan and heat source on/off independently
  • Holding a constant temperature
  • Automatic graphing of roast profiles
  • Ability to run saved roast profiles

Now instead of standing at my kitchen counter turning the air popper on and off, holding a digital temperature probe, and shaking the whole roaster to circulate and cool the beans between "on" cycles, I can simply control all of these conditions from my iPhone.

Screen shot of the web app user interface.

From my phone I decide when to heat the beans, when to maintain a certain temperature, how quickly or slowly to hit certain roasting stages or "cracks", and logging to make sure I can reproduce results in future runs.

Beans develop different flavors based on how quickly or slowly they go through different phases of roasting. Some beans might be better suited for quick roasts that will maintain acidic fruit flavors. Other beans might need to be roasted more slowly to bring out nutty and cocoa flavors. The moisture content of a bean will also have an effect on roasting times, as well as beans that are sourced from different regions of the world.

Next Steps

Although I'm extremely satisfied with how my roaster has turned out, there's still a lot on the to-do list.

I'm currently adding functionality to save roast profiles, so after an initial run of desired results, reproducing those results for the same batch of green beans is easy.

In the future, I'd like to build a second, bigger drum-style roaster for being able to roast larger batches at a time.

Follow my Github coffee roaster project page to keep up with any future updates. Also I would love to hear from anyone who has built similar projects of their own.

Good luck and happy roasting!