Quick dev directory cleanup tip

[Crosspost from my MSDN blog]

When cleaning up drive space, the first thing I do is remove the ‘obj’, ‘bin’, and ‘packages’ directories from my development directories. They are temporary and will be rebuilt the next time I build the related project. Because I end up with a lot of little test & sample projects that I don’t refer to often, their binaries and nuget directories are just taking up space.

The reason this is better than doing a Clean Solution is that Clean Solution only removes the outputs of the build. It doesn’t remove the nuget packages which in my case were a significant percentage of my overall dev directory space.

The old way

I used to do this with a little Windows Explorer trick – search filters. It looks like this

capture20160921110426561

“type:folder name:obj” tells explorer to find all the items that have “obj” in the name and are folders. Then I can easily “Select All” and delete. Then repeat for Bin and for Packages. (There is one caveat here that the name search is a substring search, so it will also return directories named “object” too.)

The PowerShell way

But today I got to thinking. I wanted to do that in one step. So here is a PowerShell command that will iterate from the current directory and find all child directories named ‘obj’, ‘bin’, or ‘packages’ and prompt me to delete them.

get-childitem -Directory -Recurse | where-object {$_.Name -eq ‘bin’ -or $_.Name -eq ‘obj’ -or $_.Name -eq ‘packages’} | Remove-Item –Force –Recurse

I put this in a file named clean-devfolder.ps1 and it works like a champ.

If, instead, you want to see a preview of which directories will be removed, you can add -WhatIf to the end of the whole command.

Advertisements
Posted in tips | Leave a comment

C# Zork running on Ubuntu

So I’ve finally got my VSTS builds running on Ubuntu (and Windows). I haven’t installed Linux on a PC in well over a decade but it has come a long way in that time.

  1. Installed Ubuntu 16.04 on HyperV on my Windows 10 dev box. Just boot the HyperV machine to an Ubuntu install disc. That was easy!
  2. Update the project.json file of my .NET Core console host
    1. to build a self-contained application (not required but I figured it would help prevent any deployment weirdness later on)
    2. to target windows and ubuntu so I can use the “dotnet publish” command
      "runtimes": {
        "win10-x64": {},
        "osx.10.11-x64": {},
        "rhel.7.2-x64": {},
        "ubuntu.14.04-x64": {}
      }
    3. Include my game data folder in the publish output
      "publishOptions": {
        "include": [ "GameFiles" ]
      },
  3. Updated the VSTS build to include two command line tasks to do the publish for both Windows and Ubuntu
    1. Add a Command Line Task
    2. Set the Tool to “dotnet”
    3. Set the Arguments to “publish –output $(build.artifactstagingdirectory)/publish/ubuntu.14.04-x64 –runtime ubuntu.14.04-x64 –configuration Release”capture20160814125439163

      This publish command will publish the given runtime package to the directory that will ultimately be copied to the drop folder by the rest of the process

      On this I step originally wasn’t paying attention to the dialog and put the entire command line  in the “tool” text box. That won’t work and you’ll get an error claiming the tool executable can’t be found. That took a while to chase down in my case because a lot of the older posts/forums discuss how to get the dotnet tooling installed into a VSTS Build agent – but it is now built in (Its at C:\program files\dotnet”)

      The final build process looks like this
      capture20160814130253915

  4. After a build is performed the drop folder looks like this
    capture20160814130415970
    I left in the original “Copy Files to” step in the build so that I had access to everything that was built as things progressed. But to run things I just need the folders under the “publish” directory
  5. Log in to Ubuntu and download the publish directory
    capture20160814130646617
  6. Last step is to run it
    capture20160814111814829

 

So the current overall workflow now is

  1. Update the project on Windows in Visual Studio Community 2015
  2. Check in to Github
  3. VSTS automatically builds, runs unit test, and packages
  4. Download to Windows or Ubuntu to run

Next I’m thinking “What could be better than running a 35 year old text adventure on Azure Service Fabric?”

Posted in Dev Stuff, ZMachine, Zork | Leave a comment

Using C# to build 35 year old tech is fun

I’ve always been a fan of Zork. If you’ve been playing games awhile (and I mean a whiiiile) then you probably are too. For those of you who aren’t familiar with it, its the original text adventure game. No graphics. Just type actions, and get text feedback. It created the whole genre of Interactive Fiction. You can actually still play it online

Recently I was on vacation and looking for a project and happened to spot Eric Lippert’s blog series describing his implementation of the Zork engine (called a ZMachine) in order to learn OCaml. I realized he was deciphering the mess that is the ZMachine spec that had always seemed a bit opaque to me. Its a seriously interesting bit of engineering to make an actual cross platform virtual machine that ran on the old Apple II TRS-80, Commodores, etc in the 80s. But a lot of their optimizations make for dense and confusing bit twiddling.

I got inspired and started building my own version in C# in an object oriented fashion with current C# techniques and libraries and unit tests.

Mine is called Leaflet and I just put it up on GitHub and will be updating it slowly. I have some oddball plans to see what I can do with this and still make sense. As an example, a while back I did an experiment with someone else’s implementation by adding Cortana’s voice recognition and Text to Speech to make it work on my Windows Phone.

Its C#
Its a .NET PCL (will run on Azure, Windows, and Linux!)
Its not optimized… yet
Its object oriented.
Its very much in progress…

I plan to write a series of posts here on some of the things I’ve learned along the way.

maybe someday I’ll revisit and reimplement it in F#.

Posted in Dev Stuff, Programming, Tinkering, ZMachine, Zork | Leave a comment

Do you have a Twinkie?

I want to be this guy. This guy who was behind us in line at the grocery store.

I’m standing there while the cashier rings up my cart, and I notice the guy behind me standing innocently doing nothing.
Then he picks up a Twinkie from his cart.
He starts to smile.
He starts to open the Twinkie.
He starts to grin.
He starts to eat the Twinkie.
He is in ecstasy.
He continues eating his Twinkie.
He is completely oblivious to the outside world.
He has no worries.
He has no problems.
He just has a Twinkie.
Enjoying his Twinkie.
Enjoying his life.

Posted in life | Leave a comment

Waaaaay oversimplified async/await plumbing

[cross posted from my MSDN blog]

Often, when someone asks “how does this async await stuff actually work”? There is a lot of hand waving or someone says “just use reflection and look at it” but the real compiled code is a complex recursive state machine. So I want to show a (relatively) simplified example that isn’t the real thing but is conceptually correct.

Conceptually, the way I think about it is that the compiler just breaks down my method into a series of tasks that need to be run. Lets start with a simple scenario which has a single await.

image

We kind of hide the Task by default but if you change it to look like this, you can see we aren’t awaiting the method, we are awaiting the Task returned from the method.

image

Now we can take the async/await keywords away by breaking the method up into 2 areas – the code that runs before the await, and the code that runs after the await. Because the GetCountAsync() call is async, we can’t run that “after” code immediately. We want to wait until after countTask is complete and then run the “after” code. So what actually runs looks more like this

image

As you can see, we have taken the “after” code and told it to run as a continuation of the countTask. This means that the GetCountAsync method will run and whenever it gets done, the “after” code will execute. Of course this means we have another task to deal with. The “after” code continuation provides us a new Task object to know when that is done. When that is finished we know that our entire method is finished. So we can return that final Task to the caller of our method so that they will know when we are complete.

Now this is definitely not a complete picture (e.g. we are not dealing with looping or exceptions or managing the UI thread). But I find this a pretty compelling way to demonstrate a couple of key principles.

  1. When you use the async keyword, your method becomes asynchronous because it gets broken into Tasks based on the await keywords in use.
  2. Your async method will return before it completes. Yes, that’s the whole point and is obvious to some people, but here it is shown explicitly because in the final example above, our method isn’t doing any real work except creating the Task chain and then returning.
  3. The code before the first await always runs synchronously. There is no Task management going on until you hit the first await. So if you have a 10 second operation before the first await, the method will take 10 seconds and then hook up the task chain and return.
  4. This is why your callstack looks different when debugging. The “after” code is not being called from DoWorkWithoutAwait(). It is being called directly from the .NET Task infrastructure as a continuation of the previous task.

Taking that a bit further

Lets take that concept and apply it to a more realistic method which contains  multiple awaits.

image

In the same steps, we can think of this as dividing our method up into a number of intermediate Tasks separated by the await keywords.

image

Again, this just demonstrates those same points that I see some developers forget or struggle with. When I see a series of await keywords, I’m thinking about how that method gets broken down into individual tasks – not how it is going to run as a single unit. This helps me remember that even if this ends up running on a single thread, the fact that is gets chunked up means that other scheduled code can potentially run in between my method’s various sections. You can also see this concept graphically in this previous post.

 

Additional files

Posted in Windows Phone | Tagged , , , | Leave a comment

Tasks are (still) not threads and async is not parallel

[cross posted from my MSDN blog]

I talk to a lot of developers who are either new to .NET or are moving from an older version to the newer platform and tools. As such I’m always trying to think of new ways to describe the nature of Tasks vs Threads and async vs parallel. Modern .NET development is steeped in the async/await model and async/await is built on the Task model. So understanding these concepts is key to long term success building apps in .NET.

In order to help visualize this I built a simple WPF application that displays a chart of an application’s activity. I want to display some of the potential variations in behavior of what appear to be a simple set of async tasks.

Take the following method

start code

This is a simple event handler which is going to call 3 asynchronous methods and then wait for all three to complete and then print out how long it took to do the whole operation. How many threads are there? Some people will assume only 1, some will assume 3. Like everything in software, it depends.

The DoWorkAsync() method just runs a loop. Each time around the loop, it will do some sort of asynchronous task and then do some busy “work” for some amount of time, then it will draw a rectangle on the screen representing the time that it spent doing that “work.” (This is analogous to making a web service call and then doing some local processing on the data returned from the service.) In this way we can easily see (a) when the work is being performed, and (b) whether the work overlaps with other task’s work. If it does, then we are running on multiple threads. The work we are concerned with is our code (e.g. the local data processing) – not the thing being waited on (the async web service), so each bar in the app will represent the local processing.

no-async code

The first case is the simplest, the method is marked as async but we really don’t have any asynchronous work going on inside it. In this case, the method is actually going to actually run to completion synchronously, meaning that the tasks above (in the event handler) will all be complete immediately upon creation and there wont be any new threads. They will each just run one right after the other.

no-async

In this image, the vertical axis is time. So first the red task ran, then the blue task ran, then the green task ran. It took 5 seconds total because 5 loops * (200ms + 300ms+ 500ms) = 5 seconds. You can also just make out the faint white lines of each individual iteration of the loops. But only 1 thread was used to run all three tasks.

Now lets make one change. Add an asynchronous operation where the //TODO is. Typically this might be a web service call or reading from a file, but for our purposes we will just use Task.Delay()

regular-async code

The only change here is the addition of the Task.Delay to the beginning of the loop. This will cause the method to not complete synchronously because it is going to do an asynchronous wait at the start of every iteration of the loop (simulating our async service call wait). Now look at the result.

regular-async

It still took about the same amount of time, but the iterations are interleaved. They are not overlapping each other though because we are still on the same thread. When you “await” an asynchronous task, the rest of your method will continue running in the same context that it started on. In WPF, that context is the UI thread. So while the red task is doing its delay, the blue and green tasks are running on the UI thread. When the red task is done delaying and wants to continue, it cant until the UI thread becomes available so it just queues up until its turn comes back around.

(Also notice that we didn’t add 1.5 seconds (100ms * 5 iterations * 3 tasks) to the total operation time. That’s the async benefit, we were able to overlap the waiting time of one task with the work time of other tasks by sharing the UI thread while we were waiting.)

But sometimes, this interleaving doesn’t happen. What if you have an asynchronous task that finishes so fast, it might as well be synchronous?

async-completed synchronously code

When the async plumbing goes to work, it first checks to see if the async operation is completed.  Task.Delay(0) will be completed immediately, and as such will be treated like a synchronous call.

async-completed synchronously

Well that puts us back to where we started. All 5 red iterations happen first because there is no asynchronous work to wait on.

Everything happens on the same thread context unless you tell it not to. Enter ConfigureAwait().

async-config await false code

ConfigureAwait(false) tells the async plumbing to ignore the thread context, and just continue on any old thread it wants. This means that as soon as the red task is done with its delay, it doesn’t have to wait for the UI thread to be available. It runs on a random threadpool thread.

async-config await false

Two things to notice here. First, since they are not bound to only running on the UI thread, they are running overlapped at the same time on multiple threads. Second, it only took 3 seconds to complete all three because they were able to fan out across multiple threads. (You can explicitly see the Task.Delays here because they are the white gaps between each bar)

Now what happens if we combine Task.Delay(0) with ConfigAwait(false)?

async-completed synchronously config await false code

Now we have a async task that will actually complete synchronously, but we are telling it not to bother with affinity for the threading context.

async-completed synchronously config await false

Completed synchronously wins. If the task completes synchronously already, then the async plumbing doesn’t come into play.

Summary

Look at this from the perspective of the original event handler above. The event handler has absolutely no idea whether its tasks are going to run on one thread or multiple threads. All it knows is that it has requested 3 potentially asynchronous tasks to be completed. The underlying implementation will determine whether additional threads come in to play or not. And whether you have multiple threads determines whether you run in parallel or not. So you need to be prepared for either behavior when writing and debugging your app because in the end, it just depends.

(Side note: The parallel behavior above is a side effect of the async/await thread context affinity in the WPF task scheduler. It is not guaranteed and the behavior may vary depending with different task schedulers. It should not be relied upon as a method to create other threads. If you require something to run in parallel, use Task.Run())

The example project used here is available in my GitHub repository.

Posted in Dev Stuff, Programming | 2 Comments

Visual Studio Tip #7: Whole line editing

[cross posted from my MSDN blog]

OK here is a quick simple one. How do I move or edit entire lines of code?

#1 Just don’t select anything.

If you don’t have anything selected in your code window then the commands for copy, cut and paste work as if the entire line of code was selected.

So if you need to quickly duplicate a line, don’t bother trying to get a whole line selected. Just put the cursor on the line then type Ctrl+C, Ctrl+V and you’ll see the new line pasted below the old line.

#2 Just move the line. Put your cursor on the line in question and use Alt+UpArrow or Alt+DownArrow to move the entire line up or down in your file.

This post is part of a series of Visual Studio tips. The first post in the series contains the whole list.

Posted in Windows Phone | Leave a comment