From time to time you will find yourself in the need of slicing up large chunks of bites into smaller chunks of consumable bites.

Since I have been unable to find such a method in the .NET framework, I decided to include one on the EnumerableUtility class found in the Cuemon.Collections.Generic namespace. The name for the method was carefully chosen to be Chunk, and consist of two overloads; both accepting an IEnumerable<T> where the chunk size is defaulted to 128 elements and the other with a custom size.

To see the Chunk method in action, have a look at Figure 1. For simplicity i set a chunk size to 255 and a workload count to 4096. This means that our workload consists of 4096 elements which we will process 255 elements at time. This leaves us with a workload of 17 runs in total.


[TestClass]
public class ChunkExample
{
    [TestMethod]
    public void TestChunk()
    {
        int chunkSize = 255;
        int workload = 4096;
        int expectedWorkloadIterations = (workload + chunkSize - 1 ) / chunkSize;
        IEnumerable<int> simulatedWorkload = new List<int>(EnumerableUtility.Range(0, workload));

        Debug.WriteLine("Work to process: {0} items.", workload);
        Debug.WriteLine("Expected iterations for work: {0} runs.", expectedWorkloadIterations);

        int actualWorkloadIterations = 0;
        while (((List<int>)simulatedWorkload).Count > 0)
        {
            IEnumerable<int> chunkedWorkload = EnumerableUtility.Chunk(ref simulatedWorkload, chunkSize);
            IEnumerator<int> chunkedWorkloadEnumerator = chunkedWorkload.GetEnumerator();
            int countOfChunkedElements = EnumerableUtility.Count(chunkedWorkload);

            Assert.IsTrue(countOfChunkedElements <= chunkSize);

            int enumeratorCount = 0;
            while (chunkedWorkloadEnumerator.MoveNext())
            {
                enumeratorCount++;
            }
            actualWorkloadIterations++;

            Assert.IsTrue(enumeratorCount <= chunkSize);

            Debug.WriteLine("Remaining work to process: {0} items.", ((List<int>)simulatedWorkload).Count);
        }

        Assert.IsTrue(expectedWorkloadIterations == actualWorkloadIterations);
        Debug.WriteLine("Actual iterations for work: {0} runs.", actualWorkloadIterations);
    }
}

Figure 1: A simple test method that utilizes the Chunk method found on the EnumerableUtility class.




When executed, the test method should produce this debug output:




Debug Trace:
Work to process: 4096 items.
Expected iterations for work: 17 runs.
Remaining work to process: 3841 items.
Remaining work to process: 3586 items.
Remaining work to process: 3331 items.
Remaining work to process: 3076 items.
Remaining work to process: 2821 items.
Remaining work to process: 2566 items.
Remaining work to process: 2311 items.
Remaining work to process: 2056 items.
Remaining work to process: 1801 items.
Remaining work to process: 1546 items.
Remaining work to process: 1291 items.
Remaining work to process: 1036 items.
Remaining work to process: 781 items.
Remaining work to process: 526 items.
Remaining work to process: 271 items.
Remaining work to process: 16 items.
Remaining work to process: 0 items.
Actual iterations for work: 17 runs.

Figure 2: The output of the test in Figure 1.



I hope you liked this little peek into one of the many day-to-day helper methods found through out the Cuemon assembly family. Happy coding!