Eureka!


Finally! And this is not just you saying or feeling it - it is also me, myself and I acknowledging it. A hectic 2015 has unfortunately postponed the Cuemon .NET Framework until today; June 1st. 2015.



So, within the next 24 hours, Cuemon .NET Framework having an assembly version of 3.0.2015.150 and a file version of 3.0.2015.1500 will be released ... as Open Source under the MIT License and hosted on Codeplex.

This was the logical next step both for me as a developer - and for the project itself having some different approaches to present the world of co-developers. Yes, not everything is perfect and some areas are actually downright embarrassing. Nevertheless hey; opening up my code to the world might help improve these areas ... and meanwhile, we can just call it Omen Source ;-)

Stay tuned - I am excited to present the hard work to all of you guys!

All the best,

MIchael Mortensen, 
aka Gimlichael


What's cooking in 2015?


So, it's been a rather long time since I last posted an update - and the reason was simple; a well deserved vacation! However, now we write 2015, and here's what cooking.


Cuemon .NET Framework 3.0 is currently being prepared with new functionality as well as kernel-wise optimizations. Also, and what is probably the most important, Cuemon will go open-source either within this month (January) or doing February 2015 on Codeplex. I am still working out the proper way to migrate from my TFS project to GitHub; looking forward to say "fork me" :-)

Last but not least (and as requested) I will start to write more articles on this blog on how to use Cuemon in your daily work. That means working examples that are more or less copy-paste friendly.

Stay tuned - and happy coding to you all.

Too much to digest? Slice it up in chunks!


From time to time you will find yourself in the need of slicing up large chunks of bites into smaller chunks of consumable bites.

Since I have been unable to find such a method in the .NET framework, I decided to include one on the EnumerableUtility class found in the Cuemon.Collections.Generic namespace. The name for the method was carefully chosen to be Chunk, and consist of two overloads; both accepting an IEnumerable<T> where the chunk size is defaulted to 128 elements and the other with a custom size.

To see the Chunk method in action, have a look at Figure 1. For simplicity i set a chunk size to 255 and a workload count to 4096. This means that our workload consists of 4096 elements which we will process 255 elements at time. This leaves us with a workload of 17 runs in total.


[TestClass]
public class ChunkExample
{
    [TestMethod]
    public void TestChunk()
    {
        int chunkSize = 255;
        int workload = 4096;
        int expectedWorkloadIterations = (workload + chunkSize - 1 ) / chunkSize;
        IEnumerable<int> simulatedWorkload = new List<int>(EnumerableUtility.Range(0, workload));

        Debug.WriteLine("Work to process: {0} items.", workload);
        Debug.WriteLine("Expected iterations for work: {0} runs.", expectedWorkloadIterations);

        int actualWorkloadIterations = 0;
        while (((List<int>)simulatedWorkload).Count > 0)
        {
            IEnumerable<int> chunkedWorkload = EnumerableUtility.Chunk(ref simulatedWorkload, chunkSize);
            IEnumerator<int> chunkedWorkloadEnumerator = chunkedWorkload.GetEnumerator();
            int countOfChunkedElements = EnumerableUtility.Count(chunkedWorkload);

            Assert.IsTrue(countOfChunkedElements <= chunkSize);

            int enumeratorCount = 0;
            while (chunkedWorkloadEnumerator.MoveNext())
            {
                enumeratorCount++;
            }
            actualWorkloadIterations++;

            Assert.IsTrue(enumeratorCount <= chunkSize);

            Debug.WriteLine("Remaining work to process: {0} items.", ((List<int>)simulatedWorkload).Count);
        }

        Assert.IsTrue(expectedWorkloadIterations == actualWorkloadIterations);
        Debug.WriteLine("Actual iterations for work: {0} runs.", actualWorkloadIterations);
    }
}

Figure 1: A simple test method that utilizes the Chunk method found on the EnumerableUtility class.




When executed, the test method should produce this debug output:




Debug Trace:
Work to process: 4096 items.
Expected iterations for work: 17 runs.
Remaining work to process: 3841 items.
Remaining work to process: 3586 items.
Remaining work to process: 3331 items.
Remaining work to process: 3076 items.
Remaining work to process: 2821 items.
Remaining work to process: 2566 items.
Remaining work to process: 2311 items.
Remaining work to process: 2056 items.
Remaining work to process: 1801 items.
Remaining work to process: 1546 items.
Remaining work to process: 1291 items.
Remaining work to process: 1036 items.
Remaining work to process: 781 items.
Remaining work to process: 526 items.
Remaining work to process: 271 items.
Remaining work to process: 16 items.
Remaining work to process: 0 items.
Actual iterations for work: 17 runs.

Figure 2: The output of the test in Figure 1.



I hope you liked this little peek into one of the many day-to-day helper methods found through out the Cuemon assembly family. Happy coding!


XML the smart way - and beyond!


Well, it's no secret - I am a big fan of XML and the possibilities it brings to a digital connected world. Yes, it may be fat - but if the consuming client support GZIP or similar - that is a big fat-loss. Still not convinced? Well, you can always transform it to a more lightweight candidate, such as JSON. That's how versatile XML is.






Speaking of versatile, there are many ways to create XML in the .NET Framework. Ranging from ways of a simple string to a StringBuilder to XmlDocument to XElement to XmlWriter to many, many more. Just Google xml .net followed by your favorite syntax, eg. c#, and several millions result is served to you.

Personally, I have always favored the XmlWriter approach, as it is as close to the core as it gets without overhead of the more popular XElement and the likes.

A typical way to use the aforementioned XmlWriter could be like this:
public Stream GetExmapleXml()
{
    Stream output = null;
    MemoryStream tempOutput = null;
    try
    {
        tempOutput = new MemoryStream();
        using (XmlWriter writer = XmlWriter.Create(tempOutput))
        {
            // build your XML here
            writer.WriteStartElement("Xml");
            writer.WriteString("A very simple XML representation - and yes, I could just have used writer.WriteElementString(..) - but hey - it's my example :-)");
            writer.WriteEndDocument();
            writer.Flush();
        }
        tempOutput.Flush();
        tempOutput.Position = 0;
        output = tempOutput;
        tempOutput = null;
    }
    finally
    {
        if (tempOutput != null) { tempOutput.Dispose(); }
    }
    return output;
}

Figure 1: A more common way of writing XML.


This would return a Stream holding this XML:

<?xml version="1.0" encoding="UTF-8"?>
<Xml>A very simple XML representation - and yes, I could just have used writer.WriteElementString(..) - but hey - it's my example :-)</Xml>

Figure 2: The XML output as produced by Figure 1, Figure 3, Figure 4 and Figure 5.



Although very performance friendly and to some extent easy on the eye, I decided to provide a more generic and simple way of achieving the same using the Cuemon.Xml assembly. In this assembly you will find the XmlWriterUtility class that (in the writing of this entry) consist of two methods; CreateSettings and CreateXml.



Since my primary goal is to be backward compatible with .NET 2.0 SP1, the CreateXml method can take up to 10 arguments using an Act delegate, which has the same characteristica as the Action delegate found in .NET 4.0. The default XML settings (should you choose not to provide a XmlWriterSettings) are the one found in the parameter less CreateSettings method.



Anyway, have a look at the same example as above - just the Cuemon way in three different forms:



public Stream GetExmapleXml()
{
    return XmlWriterUtility.CreateXml(GetExmapleXmlImpl);
}

public void GetExmapleXmlImpl(XmlWriter writer)
{
    writer.WriteStartElement("Xml");
    writer.WriteString("A very simple XML representation - and yes, I could just have used writer.WriteElementString(..) - but hey - it's my example :-)");
    writer.WriteEndDocument();
}

Figure 3: My preferred way of using the implementation - but hey - I am dinosaur in this fancy Lambda world.




public Stream GetExmapleXmlDelegate()
{
    return XmlWriterUtility.CreateXml(delegate(XmlWriter writer)
                                          {
                                              writer.WriteStartElement("Xml");
                                              writer.WriteString("A very simple XML representation - and yes, I could just have used writer.WriteElementString(..) - but hey - it's my example :-)");
                                              writer.WriteEndDocument();
                                          });
}

Figure 4: Same result as in Figure 3 - just using an anonymous method.





public Stream GetExmapleXmlLambda()
{
    return XmlWriterUtility.CreateXml(writer =>
        {
            writer.WriteStartElement("Xml");
            writer.WriteString("A very simple XML representation - and yes, I could just have used writer.WriteElementString(..) - but hey - it's my example :-)");
            writer.WriteEndDocument();
        });
}

Figure 5: Same result as in Figure 3 and 4 - just using lamda expression as supported by .NET 3.0.



There you have it - a smart and easy way of creating versatile XML that can be used in all sorts of scenarios. Want a string representation of the Stream? You asked for it - just use the ConvertUtility.ToString(..) like this:




ConvertUtility.ToString(GetExmapleXml());



I hope you enjoyed this little introduction to the XmlWriterUtility class - and can see and unleash the potential of it. As always - happy coding - and thanks for stopping by.

The dark side of the Cloud; network connectivity issues and service unavailability


In these days where everything is moving to "the cloud", new challenges arises that most of us where spared from before; temporary condition such as network connectivity issues or simply a service unavailability. The technical term for this is called Transient Fault Handling and one you must be conversant with.

On MSDN there is a very informative and good background information for Transient Fault Handling which i strongly encourage you to read. While you are there, you could consider if the comprehensive work of the Enterprise Library from version 5.0 suits your needs in what has been named the Transient Fault Handling Application Block.

If it seems a big over the top for your solution, you could consider the static TransientFaultUtility class found in the Cuemon namespace; it is fully compatible with cloud provides such as Windows Azure. It has several overloads for invoking a fault sensitive method, and will continue until the operation is successful, the amount of retry attempts has been reached, or a failed operation is not considered related to a transient fault condition.

The minimum required parameters for invoking a transient fault protected method is an integer specifying retryAttempts, a function delegate that will determine if isTransientFault and last but not least; a function delegate/action delegate pointing to the faultSensitiveMethod.

To see this in action, have a look at Figure 1. What we do here is simply throwing an HttpException should we encounter a HTTP 502. This can easily be extended to the ones listed in the IsTransientFault callback method. Otherwise we just write some debug information. Figure 2 shows how to consume the TransientFaultExample class and is intentionally set to fail in the first run. Figure 3 shows the Debug Trace.


public class TransientFaultExample
{
    public TransientFaultExample()
    {
        NetHttpUtility.DefaultHttpTimeout = TimeSpan.FromSeconds(15);
    }

    public void OpenWebsite(Uri location)
    {
        using (HttpWebResponse response = NetHttpUtility.HttpGet(location))
        {
            if (response.StatusCode == HttpStatusCode.BadGateway) { throw new HttpException(502, response.StatusDescription); }
            Debug.WriteLine("Status code in response was {0} - {1}.", (int)response.StatusCode, response.StatusDescription);
            Debug.WriteLine("The headers of the response was {0}.", ConvertUtility.ToDelimitedString(response.Headers.AllKeys, ", ", HeaderConverter, response.Headers) as object);
        }
    }

    private string HeaderConverter(string header, WebHeaderCollection headers)
    {
        return string.Format("{0}: {1}", header, headers[header] ?? "null");
    }

    public bool IsTransientFault(Exception exception)
    {
        HttpException httpException = exception as HttpException;
        if (httpException != null)
        {
            switch (httpException.GetHttpCode())
            {
                case 404:
                case 408:
                case 410:
                case 500:
                case 502:
                case 503:
                case 504:
                    return true;
                default:
                    return false;
            }
        }
        return (exception.Message.IndexOf("timed out", StringComparison.OrdinalIgnoreCase) >= 0);
    }
}

Figure 1: A simple TransientFaultExample class that can easily be rewritten to more real-life scenarios


[TestClass]
public class TransientFaultExampleTest
{
    [TestMethod]
    public void TransientFault()
    {
        TransientFaultExample transient = new TransientFaultExample();

        try
        {
            TransientFaultUtility.ExecuteAction(5, transient.IsTransientFault, transient.OpenWebsite, new Uri("http://www.google.com:88/"));
        }
        catch (TransientFaultException ex)
        {
            Debug.WriteLine("TransientFaultException was thrown (which is good): {0}", ConvertUtility.ToString(ex, Encoding.Default, true) as object);
        }

        try
        {
            TransientFaultUtility.ExecuteAction(5, transient.IsTransientFault, transient.OpenWebsite, new Uri("http://www.google.com/"));
        }
        catch (TransientFaultException ex)
        {
            Debug.WriteLine("TransientFaultException was thrown (which is not so good - for Google at least): {0}", ConvertUtility.ToString(ex, Encoding.Default, true) as object);
            Assert.Fail();
        }
    }
}

Figure 2: Consumes the class defined in Figure 1



Debug Trace:

TransientFaultException was thrown (which is good): TransientFaultException (Cuemon)
Source:
    Cuemon
Message:
    The amount of retry attempts has been reached.
Data:
    Key: Attempts
    Value: 5
    Key: RecoveryWaitTimeInSeconds
    Value: 21
    Key: TotalRecoveryWaitTimeInSeconds
    Value: 56
InnerException [of TransientFaultException]:
    TimeoutException (System)
Source:
    Cuemon
Message:
    The operation has timed out.

Status code in response was 200 - OK.
The headers of the response was Cache-Control: private, max-age=0, Content-Type: text/html; charset=ISO-8859-1, Date: Thu, 25 Apr 2013 01:09:52 GMT, Expires: -1, Set-Cookie: PREF=ID=fd0ca0865d752f5b:FF=0:TM=1366852192:LM=1366852192:S=KNM_xhwUAUmLa1-f; expires=Sat, 25-Apr-2015 01:09:52 GMT; path=/; domain=.google.dk,NID=67=vYXlkUfWQ_paZ7fdrkXaq2gmgUati-Y3FfiPzpRLQTTWn7lQgWowKZgE53z4_D1G04SmEk0N_4YdaUKC2RZkajhrCZ69QrCRHKBumdejJg4Z2MKak7fUF0QUbL7nKf3F; expires=Fri, 25-Oct-2013 01:09:52 GMT; path=/; domain=.google.dk; HttpOnly, P3P: CP="This is not a P3P policy! See http://www.google.com/support/accounts/bin/answer.py?hl=en&answer=151657 for more info.", Server: gws, X-XSS-Protection: 1; mode=block, X-Frame-Options: SAMEORIGIN, Transfer-Encoding: chunked.
Figure 3: The output of the test in Figure 2

If you don't like the default recovery wait time this can easily be added using one of the overloads on the TransientFaultUtility class. For your convenience, I have included the default implementation in Figure 4.


/// <summary>
/// Specifies the amount of time to wait for a transient fault to recover gracefully before trying a new attempt.
/// </summary>
/// <param name="currentAttempt">The current attempt.</param>
/// <returns>A <see cref="TimeSpan"/> that defines the amount of time to wait for a transient fault to recover gracefully.</returns>
/// <remarks>Default implementation is <see cref="RecoveryWaitTime"/> + 2^ to a maximum of 5; a total of 5 (default) + 32 = 37 seconds.</remarks>
public static TimeSpan RecoveryWaitTime(int currentAttempt)
{
    TimeSpan sleep = DefaultRecoveryWaitTime;
    sleep = sleep.Add(TimeSpan.FromSeconds(Math.Pow(2, currentAttempt > 5 ? 5 : currentAttempt)));
    return sleep;
}

Figure 4: The default implementation of the function delegate recoveryWaitTime



I hope this little introduction have inspired you to master Transient Fault Handling one way or another. Happy coding.