<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Marius Bancila's Blog &#187; Parallel Programming</title>
	<atom:link href="http://mariusbancila.ro/blog/category/it/software/parallel-programming/feed/" rel="self" type="application/rss+xml" />
	<link>http://mariusbancila.ro/blog</link>
	<description>Sharing my opinions and ideas!</description>
	<lastBuildDate>Fri, 06 Apr 2012 13:45:55 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.3.1</generator>
		<item>
		<title>C++ Renaissance at Microsoft</title>
		<link>http://mariusbancila.ro/blog/2011/06/20/cpp-renaissance-at-microsoft/</link>
		<comments>http://mariusbancila.ro/blog/2011/06/20/cpp-renaissance-at-microsoft/#comments</comments>
		<pubDate>Mon, 20 Jun 2011 08:59:27 +0000</pubDate>
		<dc:creator>Marius Bancila</dc:creator>
				<category><![CDATA[C++]]></category>
		<category><![CDATA[Parallel Programming]]></category>
		<category><![CDATA[Tools]]></category>
		<category><![CDATA[Windows Programming]]></category>
		<category><![CDATA[renaissance]]></category>
		<category><![CDATA[resources]]></category>
		<category><![CDATA[VisualStudio]]></category>

		<guid isPermaLink="false">http://mariusbancila.ro/blog/?p=1019</guid>
		<description><![CDATA[At the beginning of this year, Microsoft announced a &#8220;C++ renaissance&#8221;. Quoting from the description of a Channel 9 video with Craig Symonds and Mohsen Agsen: C++ is currently undergoing a renaissance. This means that, by definition, the language, compilers and compositional tooling are evolving and coalescing into a state that maximizes native developer efficiency, [...]]]></description>
			<content:encoded><![CDATA[<p>At the beginning of this year, Microsoft announced a &#8220;C++ renaissance&#8221;. Quoting from the description of a <a href="http://channel9.msdn.com/">Channel 9</a> video with Craig Symonds and Mohsen Agsen:</p>
<blockquote><p>C++ is currently undergoing a renaissance. This means that, by definition, the language, compilers and compositional tooling are evolving and coalescing into a state that maximizes native developer efficiency, productivity, and creativity across hardware and software domains.</p></blockquote>
<p>Everybody agrees that Microsoft made C++ a sort of second class citizen in the past years, while the company invested a lot in the .NET framework. Many developers have switched from native development to managed (.NET) simply because it offers a more productive environment. And the postponing of the ISO standard committee in releasing the new C++0x standard only made things worse. </p>
<p>However, with the completion of the new C++ standard this year, Microsoft, apparently, plans to change that, and make C++ again appealing to developers. They already made C++0x features available in the VS2010 C++ compiler and are working on implementing most of the rest for Visual Studio vNext. They are also investing in tools (now labeled Application Lifecycle Management), and for instance are bringing intellisence to C++/CLI. One of the most important areas of development is parallelism, where they are developing the PPL and Agents libraries and now the C++ AMP that they just announced. And also recently the Kinect for Windows SDK beta that provides Kinect capabilities to developers who build applications with C++ (and other laguanges). And in the mean time <a href="http://blogs.msdn.com/b/jasonz/archive/2011/06/06/welcome-erich-gamma-to-the-visual-studio-team.aspx">they hired Erich Gamma in the Visual Studio team</a>.</p>
<p>But this is not enough in my opinion. Improvements in language and tools are an important part, but not everything. It is equally necessary for Microsoft to evangelize it, using any necessary means. Unless they can spread the word, the work might pass unnoticed. To be honest, I was very reluctant about this part, half an year ago, when they announced the &#8220;renaissance&#8221;. However, looking back at what they done I&#8217;d say they are on the right track. Of course, there is still a lot of work to match the &#8220;advertising&#8221; effort put into .NET. But right now C++ is getting more attention at conferences such as PDC or TechEd, or their publishing assets, such as Channel 9, MSDN or their team blogs. So I tried to assemble a collection of videos, blogs, books and code samples related to C++ or native development that they published since the announcement of the renaissance. So far it looks good, in my opinion.</p>
<p>Channel 9<br />
<a href="http://channel9.msdn.com/Shows/Going+Deep/E2E-Herb-Sutter-and-Erik-Meijer-Perspectives-on-C">E2E: Herb Sutter and Erik Meijer &#8211; Perspectives on C++</a><br />
<a href="http://channel9.msdn.com/Shows/Going+Deep/Craig-Symonds-and-Mohsen-Agsen-C-Renaissance">Craig Symonds and Mohsen Agsen: C++ Renaissance</a><br />
<a href="http://channel9.msdn.com/posts/Windows-7-Taskbar-Integration-for-MFC-Applications">Windows 7 Taskbar Integration for MFC Applications</a><br />
<a href="http://channel9.msdn.com/Blogs/Charles/Tony-Goodhew-VC-Developer-Communication-Questions-and-Answers">Tony Goodhew: VC++ Developer Communication &#8211; Questions and Answers</a><br />
<a href="http://channel9.msdn.com/Blogs/Charles/VC-MVP-Summit-Kate-Gregory">Talkin&#8217; C++ with Kate Gregory</a><br />
<a href="http://channel9.msdn.com/Blogs/Charles/MVP-Summit-2011-Meet-C-MVPs-Angel-PJ-Tom-and-Sheng">MVP Summit 2011: Meet C++ MVPs Angel, PJ, Tom and Sheng</a><br />
<a href="http://channel9.msdn.com/Blogs/Charles/MVP-Summit-2011-Meet-C-MVPs-Alon-Marius-Bruno-and-Jim">Talkin&#8217; C++ with Alon, Marius, Bruno, and Jim</a><br />
<a href="http://channel9.msdn.com/posts/Talkin-C-with-Boris-Jabes-C-Intellisense-Game-Development-and-Boris-Faces-His-Demons">Talkin&#8217; C++ with Boris Jabes: C++ Intellisense, Game Development, and Boris Faces His Demons</a><br />
<a href="http://channel9.msdn.com/posts/Application-Restart-and-Recovery-on-Windows-7-in-Native-Code">Application Restart and Recovery on Windows 7 in Native Code</a><br />
<a href="http://channel9.msdn.com/Shows/Going+Deep/Parallel-Programming-in-Native-Code-Tasks-and-Continuations-Part-1-of-2">Parallel Programming for C++ Developers: Tasks and Continuations, Part 1 of 2</a><br />
<a href="http://channel9.msdn.com/Shows/Going+Deep/Parallel-Programming-for-C-Developers-Tasks-and-Continuations-Part-2-of-2">Parallel Programming for C++ Developers: Tasks and Continuations, Part 2 of 2</a><br />
<a href="http://channel9.msdn.com/Shows/Going+Deep/Conversation-with-Herb-Sutter-Perspectives-on-Modern-C0x11">Conversation with Herb Sutter: Perspectives on Modern C++(0x/11)</a><br />
<a href="http://channel9.msdn.com/posts/C-ALM--Architecture-Tools-and-Unit-Testing-for-Visual-C-Developers">First Look: New ALM Tools for VC++ Developers</a><br />
<a href="http://channel9.msdn.com/Events/TechEd/NorthAmerica/2011/DEV303">Modern Native C++ Development for Maximum Productivity</a><br />
<a href="http://channel9.msdn.com/Shows/Going+Deep/Mohsen-Agsen-C-Today-and-Tomorrow">Mohsen Agsen &#8211; C++ Today and Tomorrow</a><br />
<a href="http://channel9.msdn.com/Shows/Going+Deep/Herb-Sutter-C-Questions-and-Answers">Herb Sutter: C++ Questions and Answers</a><br />
<a href="http://channel9.msdn.com/posts/AFDS-Keynote-Herb-Sutter-Heterogeneous-Computing-and-C-AMP">Herb Sutter &#8211; Heterogeneous Computing and C++ AMP</a><br />
<a href="http://channel9.msdn.com/posts/Daniel-Moth-Blazing-fast-code-using-GPUs-and-more-with-C-AMP">Daniel Moth: Blazing-fast code using GPUs and more, with C++ AMP</a><br />
<a href="http://channel9.msdn.com/Shows/Going+Deep/C9-Lectures-Stephan-T-Lavavej-Advanced-STL-1-of-n">C9 Lectures: Stephan T Lavavej &#8211; Advanced STL, 1 of n</a><br />
<a href="http://channel9.msdn.com/Shows/Going+Deep/C9-Lectures-Stephan-T-Lavavej-Advanced-STL-2-of-n">C9 Lectures: Stephan T Lavavej &#8211; Advanced STL, 2 of n</a><br />
<a href="http://channel9.msdn.com/Shows/Going+Deep/C9-Lectures-Stephan-T-Lavavej-Advanced-STL-3-of-n">C9 Lectures: Stephan T Lavavej &#8211; Advanced STL, 3 of n</a><br />
<a href="http://channel9.msdn.com/Shows/Going+Deep/C9-Lectures-Stephan-T-Lavavej-Advanced-STL-4-of-n">C9 Lectures: Stephan T Lavavej &#8211; Advanced STL, 4 of n</a><br />
<a href="http://channel9.msdn.com/Shows/Going+Deep/C9-Lectures-Stephan-T-Lavavej-Advanced-STL-5-of-n">C9 Lectures: Stephan T Lavavej &#8211; Advanced STL, 5 of n</a></p>
<p>Visual C++ Team Blog<br />
<a href="http://blogs.msdn.com/b/vcblog/archive/2011/01/05/damn-my-vc-project-is-building-slower-in-vs2010-what-do-i-do-now-a-step-by-step-guide.aspx">Grr&#8230; My VC++ Project Is Building Slower in VS2010. What Do I Do Now? (A Step by Step Guide)</a><br />
<a href="http://blogs.msdn.com/b/vcblog/archive/2011/03/03/10136696.aspx">C++/CLI IntelliSense in Visual Studio vNext</a><br />
<a href="http://blogs.msdn.com/b/vcblog/archive/2011/03/22/10143956.aspx">Exception Boundaries: Working With Multiple Error Handling Mechanisms</a><br />
<a href="http://blogs.msdn.com/b/vcblog/archive/2011/03/29/10146895.aspx">Troubleshooting Tips for IntelliSense Slowness</a><br />
<a href="http://blogs.msdn.com/b/vcblog/archive/2011/04/05/10149742.aspx">Build Related Improvement in VS2010 SP1</a><br />
<a href="http://blogs.msdn.com/b/vcblog/archive/2011/05/04/10161066.aspx">Converting An MFC Ribbon To Designer Format</a><br />
<a href="http://blogs.msdn.com/b/vcblog/archive/2011/05/26/10168887.aspx">Enforcing Correct Concurrent Access of Class Data</a></p>
<p>Parallel Programming in Native Code Blog<br />
<a href="http://blogs.msdn.com/b/nativeconcurrency/archive/2011/01/15/sorting-in-ppl.aspx">Sorting in PPL</a><br />
<a href="http://blogs.msdn.com/b/nativeconcurrency/archive/2011/01/26/how-to-pick-your-parallel-sort.aspx">How to pick your parallel sort?</a><br />
<a href="http://blogs.msdn.com/b/nativeconcurrency/archive/2011/02/21/the-concurrency-runtime-and-visual-c-2010-lambda-expressions.aspx">The Concurrency Runtime and Visual C++ 2010: Lambda Expressions</a><br />
<a href="http://blogs.msdn.com/b/nativeconcurrency/archive/2011/02/28/the-concurrency-runtime-and-visual-c-2010-automatic-type-deduction.aspx">The Concurrency Runtime and Visual C++ 2010: Automatic Type Deduction</a><br />
<a href="http://blogs.msdn.com/b/nativeconcurrency/archive/2011/03/07/the-concurrency-runtime-and-visual-c-2010-the-decltype-type-specifier.aspx">The Concurrency Runtime and Visual C++ 2010: The decltype Type Specifier</a><br />
<a href="http://blogs.msdn.com/b/nativeconcurrency/archive/2011/03/14/the-concurrency-runtime-and-visual-c-2010-rvalue-references.aspx">The Concurrency Runtime and Visual C++ 2010: Rvalue References</a><br />
<a href="http://blogs.msdn.com/b/nativeconcurrency/archive/2011/03/24/the-concurrency-runtime-and-visual-c-2010-transporting-exceptions-between-threads.aspx">The Concurrency Runtime and Visual C++ 2010: Transporting Exceptions between Threads</a><br />
<a href="http://blogs.msdn.com/b/nativeconcurrency/archive/2011/03/23/building-responsive-gui-applications-with-ppl-tasks.aspx">Building Responsive GUI Applications with PPL Tasks</a></p>
<p>MSDN Magazine<br />
<a href="http://msdn.microsoft.com/en-us/magazine/gg650659.aspx">Writing a Debugging Tools for Windows Extension</a><br />
<a href="http://msdn.microsoft.com/en-us/magazine/hh148143.aspx">Writing a Debugging Tools for Windows Extension, Part 2: Output</a><br />
<a href="http://msdn.microsoft.com/en-us/magazine/hh205613.aspx">Writing a Debugging Tools for Windows Extension, Part 3: Clients and Callbacks</a><br />
<a href="http://msdn.microsoft.com/en-us/magazine/hh205645.aspx">Agile C++ Development and Testing with Visual Studio and TFS</a></p>
<p>Books &#038; Publications<br />
<a href="http://parallelpatternscpp.codeplex.com/">Parallel Programming with Microsoft Visual C++</a><br />
<a href="http://paper.li/visualc/news">The Visual C++ Weekly</a></p>
<p>Code &#038; Samples<br />
<a href="http://archive.msdn.microsoft.com/concrtextras">Code samples for the Concurrency Runtime and Parallel Pattern Library in Visual Studio 2010</a><br />
<a href="http://code.msdn.microsoft.com/Bing-Maps-Trip-Optimizer-003021ae">Bing Maps Trip Optimizer</a><br />
<a href="http://msdn.microsoft.com/en-us/library/ff708696.aspx">Hilo: Developing C++ Applications for Windows 7</a><br />
<a href="http://1code.codeplex.com/">All-in-One Code Framework</a></p>
]]></content:encoded>
			<wfw:commentRss>http://mariusbancila.ro/blog/2011/06/20/cpp-renaissance-at-microsoft/feed/</wfw:commentRss>
		<slash:comments>10</slash:comments>
		</item>
		<item>
		<title>C++ Accelerated Massive Parallelism</title>
		<link>http://mariusbancila.ro/blog/2011/06/16/c-accelerated-massive-parallelism/</link>
		<comments>http://mariusbancila.ro/blog/2011/06/16/c-accelerated-massive-parallelism/#comments</comments>
		<pubDate>Thu, 16 Jun 2011 07:55:05 +0000</pubDate>
		<dc:creator>Marius Bancila</dc:creator>
				<category><![CDATA[Parallel Programming]]></category>
		<category><![CDATA[Tools]]></category>
		<category><![CDATA[Visual Studio]]></category>
		<category><![CDATA[C++]]></category>
		<category><![CDATA[concurrency]]></category>
		<category><![CDATA[library]]></category>
		<category><![CDATA[parallel]]></category>
		<category><![CDATA[VisualStudio]]></category>

		<guid isPermaLink="false">http://mariusbancila.ro/blog/?p=1011</guid>
		<description><![CDATA[Microsoft has recently announced at the AMD Fusion Developer Summit the introduction of a new technology called C++ Accelerated Massive Parallelism or shortly C++ AMP that helps C++ developers use the GPU for parallel programming. The new technology will be part of Visual C++, integrated with full support (edit, build, debug, profile) in the next [...]]]></description>
			<content:encoded><![CDATA[<p>Microsoft has recently announced at the AMD Fusion Developer Summit the introduction of a new technology called <strong>C++ Accelerated Massive Parallelism</strong> or shortly <strong>C++ AMP</strong> that helps C++ developers use the GPU for parallel programming. The new technology will be part of Visual C++, integrated with full support (edit, build, debug, profile) in the next version of Visual Studio. It is built in modern C++ on top of DirectX. It will provide an STL-like library, part of the Concurrency namespace, delivered in a amp.h header.</p>
<p>S.Somasegar (Senior Vice President of the Developer Division) said:</p>
<blockquote><p>By building on the Windows DirectX platform, our implementation of C++ AMP allows you to target hardware from all the major hardware vendors. We expect that it will be part of the next Visual C++ compiler and fully integrated in the next release of Visual Studio experience. </p></blockquote>
<p>You can read more about it here:</p>
<ul>
<li><a href="http://blogs.msdn.com/b/somasegar/archive/2011/06/15/targeting-heterogeneity-with-c-amp-and-ppl.aspx">Targeting Heterogeneity with C++ AMP and PPL</a></li>
<li><a href="http://www.danielmoth.com/Blog/C-Accelerated-Massive-Parallelism.aspx">C++ Accelerated Massive Parallelism</a></li>
<li><a href="http://blogs.msdn.com/b/vcblog/archive/2011/06/15/introducing-amp.aspx">Introducing C++ Accelerated Massive Parallelism (C++ AMP)</a></li>
</ul>
<p>Looking forward for the first CTP and samples.</p>
<p>UPDATE<br />
Herb Sutter introduces C++ AMP at the AMD Fusion Developer Summit 11<br />
<iframe style="height:288px;width:512px" src="http://channel9.msdn.com/posts/AFDS-Keynote-Herb-Sutter-Heterogeneous-Computing-and-C-AMP/player?w=512&#038;h=288" frameBorder="0" scrolling="no" ></iframe></p>
<p>Daniel Moth digs deeper into C++ AMP with code samples and more<br />
<iframe style="height:288px;width:512px" src="http://channel9.msdn.com/posts/Daniel-Moth-Blazing-fast-code-using-GPUs-and-more-with-C-AMP/player?w=512&#038;h=288" frameBorder="0" scrolling="no" ></iframe></p>
<p>Herb Sutter</p>
<blockquote><p>How long is your password? How long it will take a 100,000,000 GPU cores running at what, a million attempts per second to crack your password just by brute force? That where almost a kid can write that. An that is just a tiny example of how game changing this is.</p></blockquote>
]]></content:encoded>
			<wfw:commentRss>http://mariusbancila.ro/blog/2011/06/16/c-accelerated-massive-parallelism/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Parallel Programming for C++ Developers</title>
		<link>http://mariusbancila.ro/blog/2011/03/15/parallel-programming-for-c-developers/</link>
		<comments>http://mariusbancila.ro/blog/2011/03/15/parallel-programming-for-c-developers/#comments</comments>
		<pubDate>Tue, 15 Mar 2011 09:08:30 +0000</pubDate>
		<dc:creator>Marius Bancila</dc:creator>
				<category><![CDATA[C++]]></category>
		<category><![CDATA[Parallel Programming]]></category>
		<category><![CDATA[channel9]]></category>
		<category><![CDATA[concurrency]]></category>
		<category><![CDATA[parallel]]></category>
		<category><![CDATA[PPL]]></category>
		<category><![CDATA[task]]></category>

		<guid isPermaLink="false">http://mariusbancila.ro/blog/?p=874</guid>
		<description><![CDATA[Channel9 recently posted a video with the Parallel Computing Concurrency Runtime team talking, mainly, about tasks and continuations, new features to the Parallel Patterns Library. These are already available through the ConcRT Extra&#8217;s Sample Pack. You can watch the half hour interview with the team here. Besides the new stuff they shown, I particularly liked [...]]]></description>
			<content:encoded><![CDATA[<p>Channel9 recently posted a video with the <a href="http://blogs.msdn.com/b/nativeconcurrency/">Parallel Computing Concurrency Runtime team</a> talking, mainly, about <em>tasks</em> and <em>continuations</em>, new features to the Parallel Patterns Library. These are already available through the <a href="http://archive.msdn.microsoft.com/concrtextras">ConcRT Extra&#8217;s Sample Pack</a>. You can watch the half hour interview with the team <a href="http://channel9.msdn.com/Shows/Going+Deep/Parallel-Programming-in-Native-Code-Tasks-and-Continuations-Part-1-of-2">here</a>.</p>
<p>Besides the new stuff they shown, I particularly liked two things that Artur Laksberg said. The first was about the difference between <em>parallelism</em> and <em>concurrency</em>:</p>
<blockquote><p><em>Parallelism</em> is doing the same amount of work faster by utilizing multiple cores, multiple processors. <em>Concurrency</em> is understood as doing more work in the same amount of time.</p></blockquote>
<p>The other one was about threads and tasks:</p>
<blockquote><p>We want people to stop thinking about threads and start thinking in terms of independent, or not independent, units of work. You have one piece of work, you compose it with another piece of work and you have two tasks, you join together and what you get as a result is another task. And then, concurrency, as somebody said, just happens. It just happens if the runtime decides it&#8217;s beneficial for you, that it is safe to execute those two chunks, tasks, in parallel.</p></blockquote>
<p>Hopefully people will start understanding that threads are obsolete and they should be thinking in tasks.</p>
<p>UPDATE: Microsoft Technical Computing group announced yesterday the availability of a book called <a href="http://msdn.microsoft.com/en-us/library/gg675934.aspx">Parallel Programming with Microsoft Visual C++: Design patterns for Decomposition, and Coordination on Multicore Architectures</a>, describing six key patterns for data and task parallelism and how to implement them in VC++ using the Parallel Patterns Library and Asynchronous Agents Library, which shipped with Visual Studio 2010. There is also a printed version for the book. You can read more about it on <a href="http://blogs.msdn.com/b/vcblog/archive/2011/03/15/10139453.aspx">VC++ team&#8217;s blog</a>.</p>
]]></content:encoded>
			<wfw:commentRss>http://mariusbancila.ro/blog/2011/03/15/parallel-programming-for-c-developers/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
		<item>
		<title>Results of the Concurrency Questionnaire</title>
		<link>http://mariusbancila.ro/blog/2010/12/13/results-of-the-concurrency-questionnaire/</link>
		<comments>http://mariusbancila.ro/blog/2010/12/13/results-of-the-concurrency-questionnaire/#comments</comments>
		<pubDate>Mon, 13 Dec 2010 12:09:03 +0000</pubDate>
		<dc:creator>Marius Bancila</dc:creator>
				<category><![CDATA[Parallel Programming]]></category>
		<category><![CDATA[concurrency]]></category>
		<category><![CDATA[parallel]]></category>
		<category><![CDATA[questionnaire]]></category>
		<category><![CDATA[results]]></category>

		<guid isPermaLink="false">http://mariusbancila.ro/blog/?p=815</guid>
		<description><![CDATA[At the beginning of this year I have posted a short questionnaire on concurrency on my blog. I have promised to publish the results, so here they are. The number of respondents was 27 (and not everybody answered all the questions). Q: What is your role in application development? Q: How many cores has your [...]]]></description>
			<content:encoded><![CDATA[<p>At the beginning of this year I have posted a short <a href="http://mariusbancila.ro/blog/2010/03/10/questionaire-on-concurrency/">questionnaire on concurrency</a> on my blog. I have promised to publish the results, so here they are. The number of respondents was 27 (and not everybody answered all the questions).</p>
<p><strong>Q</strong>: What is your role in application development?<br />
<img alt="" src="/blog/wp-content/uploads/2010/12/question1.png" title="Question 1" class="alignnone" width="705" height="187" /></p>
<p><strong>Q</strong>: How many cores has your primary working station (PC, laptop)?<br />
<img alt="" src="/blog/wp-content/uploads/2010/12/question2.png" title="Question 2" class="alignnone" width="520" height="217" /></p>
<p><strong>Q</strong>: What type of application(s) are you developing?<br />
<strong>A</strong>: Windows and web applications, ERP, banking, internal, back-end, integrations (addins: IE, Outlook, etc.), VxWorks kernel-base applications under PPC603, Point-of-Sale, internal, Web site + ERP backend + e-commerce, web applications, Windows applications, Mac applications, control software, visualizations, scada, Linux embedded application, Windows forms, Windows .Net with WPF/WCF and Silverlight, integration technology platforms for CRM and ERP, Kernel mode drivers, filesystem drivers, graphic engines, CAD.</p>
<p><strong>Q</strong>: What programming languages are used for your application development?<br />
<img alt="" src="/blog/wp-content/uploads/2010/12/question4.png" title="Question 4" class="alignnone" width="529" height="217" /></p>
<p><strong>Q</strong>: One a scale from 1 to 5 how do you rate your level of awareness on concurrency issues?<br />
<img alt="" src="/blog/wp-content/uploads/2010/12/question5.png" title="Question 5" class="alignnone" width="337" height="149" /></p>
<p><strong>Q</strong>: On a scale from 1 to 5 how do you rate your activity in getting informed and acquiring competence on concurrency (trends, tools, etc.)?<br />
<img alt="" src="/blog/wp-content/uploads/2010/12/question6.png" title="Question 6" class="alignnone" width="337" height="150" /></p>
<p><strong>Q</strong>: What do you expect to happen to developers that won&#8217;t learn to program concurrently?<br />
<img alt="" src="/blog/wp-content/uploads/2010/12/question7.png" title="Question 7" class="alignnone" width="537" height="268" /></p>
<p><strong>Q</strong>: On a scale from 1 to 5, how would you grade the level of parallization of your application?<br />
<img alt="" src="/blog/wp-content/uploads/2010/12/question8.png" title="Question 8" class="alignnone" width="330" height="151" /></p>
<p><strong>Q</strong>: Does your application performance scale proportionally with the number of cores?<br />
<img alt="" src="/blog/wp-content/uploads/2010/12/question9.png" title="Question 9" class="alignnone" width="773" height="244" /></p>
<p><strong>Q</strong>: What libraries and tools are used in your application for concurrency?<br />
<strong>A</strong>: System.Threading, The built-in threading classes, OS support (CRT, Win32 API), VS2010, .Net Framework &#8211; Thread, ThreadPool, Task, Monitor, lock() and many own written libraries (synchronized objects, asynchron calls and functions), MFC framework, .NET framework, Qt framework, Java framework, My own using RAII, nothing special: self made priority queues and thread pools, Win32 API&#8217;s, Windows threads.</p>
<p><strong>Q</strong>: Give examples of routines in your application that are executed in worker threads.<br />
<strong>A</strong>: loading of grids which care displayed at startup in the main application window, logging, tracing, loading data (in some cases), long running tasks (computations, report generation also in some cases), HTTP lenghtly posts, Outlook inbox scan for text, a job/task manager for things that are triggered by web UI but are not required to output any result to the client, data processing, database interactions, database data loading/uploading, looped data acquisition presented in chart monitors, sattelite data scaning and sectioned for DVB-S systems, any operations where I need to start/stop, all long running operations, application request processing, connection management, I/O, Open large files, communication, file uploads, update checking.</p>
<p><strong>Q</strong>: Give examples of routines in your application that execute in the main (UI) thread and can be refactored to execute in worker threads.<br />
<strong>A</strong>: loading all application modules, SQL queries, draw rows of panels &#8211; maybe even threading each individual block per row, all data processing which take more than a few seconds and which are not dependent or influence other events on UI, I&#8217;ve been multi-threaded programming for so long, I usually design the app for multi-theading up front, long (minutes) calculations, user triggered opening of multiple files, opening of a single file, certain post-processing of data, database stuff.</p>
<p><strong>Q</strong>: Give examples of routines in your application that are parallelized.<br />
<strong>A</strong>: logging and Tracing with multiple outputs (File System, Debug Window, Activity Logs written to database), UI, background processes, sending emails, invokes an event multithreaded in an own ThreadPool, all data processing in an application running in a sattelite set top box, zapit-thread, epg-thread, timer thread, other threads controlling whichever driver involved: front panel display driver, sound driver etc., remote connection handling, data processing, 3d transformations.</p>
<p><strong>Q</strong>: Give examples of routines in your application that could be parallelized to improve performance.<br />
<strong>A</strong>: cost computation and distribution for productions chains, the rest, all timers routines should turn to threads, all routines that take more than a few seconds and locks the UI, send-request-and-wait-response logic (actually I was thinking of SEDA kind of processing), importing models from project files.</p>
]]></content:encoded>
			<wfw:commentRss>http://mariusbancila.ro/blog/2010/12/13/results-of-the-concurrency-questionnaire/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
		<item>
		<title>Questionaire on concurrency</title>
		<link>http://mariusbancila.ro/blog/2010/03/10/questionaire-on-concurrency/</link>
		<comments>http://mariusbancila.ro/blog/2010/03/10/questionaire-on-concurrency/#comments</comments>
		<pubDate>Wed, 10 Mar 2010 10:20:48 +0000</pubDate>
		<dc:creator>Marius Bancila</dc:creator>
				<category><![CDATA[Parallel Programming]]></category>
		<category><![CDATA[concurrency]]></category>
		<category><![CDATA[multithreads]]></category>
		<category><![CDATA[parallel]]></category>
		<category><![CDATA[survey]]></category>

		<guid isPermaLink="false">http://mariusbancila.ro/blog/?p=473</guid>
		<description><![CDATA[I think application development faces two challenges nowadays: 64-bit and multi-core/many-core hardware. Switching from 32 to 64 bit is just another step in the evolution of processors. There was a time when we switched from 8 to 16, and then from 16 to 32. There are problems that arise every time, but probably in 10-20 [...]]]></description>
			<content:encoded><![CDATA[<p>I think application development faces two challenges nowadays: 64-bit and multi-core/many-core hardware. Switching from 32 to 64 bit is just another step in the evolution of processors. There was a time when we switched from 8 to 16, and then from 16 to 32. There are problems that arise every time, but probably in 10-20 years we will have 128 bit platforms. On the other hand, multi-core/many-core is a different shift not only in development but also thinking. We either run one core or multiple core processors. My working stations has 8 cores (4 physical and 4 virtualized) and so does my laptop. Still, I don&#8217;t see an 8 times improvement of the applications running on these machines; yet I know 8 times I not what I should expect. But not even 4 times. For instance, the time for building from scratch the application I&#8217;ve working on with Visual Studio 2008 dropped from 20 minutes to 8 minutes; that&#8217;s a 2.5 improvement. I know that having N cores doesn&#8217;t mean that applications can run N times faster, because not everything can run in several threads, and then we have problems with resource access, synchronization and others. All these prevent applications run N times faster. But the problem is that we are not thinking in parallel. We are still used to program in a single thread; and when I say that I mean not only most developers are not used to do parallelization for boosting performance of some routines, but also many operations are run in the main (UI) thread, making that GUI freeze.</p>
<p>Therefore I would like to get some feedback from people working in applications development to get an idea about awareness, issues, solutions regarding concurrency. Please take several minutes to answer the questions in this survey.</p>
<p>I&#8217;d like to quote James Reinders, lead evangelist and Director of Marketing and Business Development for Intel Software Development Products, who said that:</p>
<blockquote><p>I am still confident that software development in 2016 will not be kind to programmers who have not learned to &#8220;Think Parallel.&#8221;</p></blockquote>
<blockquote><p>The &#8220;not parallel&#8221; era we are now exiting will appear to be a very primitive time in the history of computers when people look back in a hundred years. The world works in parallel, and it is time for computer programs to do the same.</p></blockquote>
<blockquote><p>Doing one thing at a time is &#8220;so yesterday.&#8221;</p></blockquote>
<p>The questionnaire bellow is also available <a href="https://spreadsheets.google.com/viewform?hl=en&#038;formkey=dE5mRlc2ejhXMVpaSUtPZmkzaHZkanc6MA" target="_blank">here</a>.</p>
<p><iframe src="https://spreadsheets.google.com/embeddedform?formkey=dE5mRlc2ejhXMVpaSUtPZmkzaHZkanc6MA" width="700" height="3400" frameborder="0" marginheight="0" marginwidth="0">Loading&#8230;</iframe></p>
<p>Thank you for answering the questionnaire.</p>
]]></content:encoded>
			<wfw:commentRss>http://mariusbancila.ro/blog/2010/03/10/questionaire-on-concurrency/feed/</wfw:commentRss>
		<slash:comments>2</slash:comments>
		</item>
		<item>
		<title>Helpers For Multithreading in C++</title>
		<link>http://mariusbancila.ro/blog/2010/02/01/helpers-for-multithreading-cpp/</link>
		<comments>http://mariusbancila.ro/blog/2010/02/01/helpers-for-multithreading-cpp/#comments</comments>
		<pubDate>Mon, 01 Feb 2010 20:32:17 +0000</pubDate>
		<dc:creator>Marius Bancila</dc:creator>
				<category><![CDATA[C++]]></category>
		<category><![CDATA[Parallel Programming]]></category>
		<category><![CDATA[C++ multithreads threads concurrency]]></category>

		<guid isPermaLink="false">http://mariusbancila.ro/blog/?p=466</guid>
		<description><![CDATA[On of the most important challenges nowadays in programming is concurrency. If we don&#8217;t learn to write programs that are able to run on multiple cores the progress in hardware will be pointless. But when you run multiple threads for various processing you might face the situation when you have to write over and over [...]]]></description>
			<content:encoded><![CDATA[<p>On of the most important challenges nowadays in programming is concurrency. If we don&#8217;t learn to write programs that are able to run on multiple cores the progress in hardware will be pointless. But when you run multiple threads for various processing you might face the situation when you have to write over and over again same or similar code for creating the threads, setting up the parameters for the threads, joining the threads, checking the result, cleaning-up, etc.</p>
<p>In this post I will show how you can create some helpers in C++ to simplify this process. This is not going to be a full solution, neither a solution that fits all needs, but can be a start.</p>
<p>What I would like to have is a helper class that will take care of:</p>
<ul>
<li>finding how many threads can run (considering each available core can run a thread)</li>
<li>creating and starting the threads</li>
<li>joining the threads</li>
<li>checking the result of the threads execution</li>
<li>cleaning up</li>
</ul>
<p>The class show bellow does just that.</p>
<pre name="code" class="cpp">
#include &lt;windows.h&gt;

class ThreadHelper
{
	LPVOID* m_Params;
	int m_ThreadsNo;

private:
	int GetProcessorsCount()
	{
		SYSTEM_INFO info;
		::GetSystemInfo(&amp;info);
		return info.dwNumberOfProcessors;
	}

public:
	ThreadHelper()
	{
		m_ThreadsNo = GetProcessorsCount();

		m_Params = new LPVOID[m_ThreadsNo];
		for(int i = 0; i &lt; m_ThreadsNo; ++i)
			m_Params[i] = NULL;
	}

	ThreadHelper(int threadsNo)
	{
		if(threadsNo &lt; 1)
			m_ThreadsNo = GetProcessorsCount();
		else
			m_ThreadsNo = threadsNo;

		m_Params = new LPVOID[m_ThreadsNo];
		for(int i = 0; i &lt; m_ThreadsNo; ++i)
			m_Params[i] = NULL;
	}

	~ThreadHelper()
	{
		delete [] m_Params;
	}

	int GetThreadsNo() const {return m_ThreadsNo;}
	bool SetThreadParams(int threadIndex, LPVOID lpData)
	{
		if(threadIndex &gt;= 0 &#038;&#038; threadIndex &lt; m_ThreadsNo)
		{
			m_Params[threadIndex] = lpData;
			return true;
		}

		return false;
	}

	bool Run(LPTHREAD_START_ROUTINE threadProc, BOOL startImmediatelly, DWORD timeout = INFINITE)
	{
		bool success = false;

		HANDLE* hThreads = new HANDLE[m_ThreadsNo];
		DWORD* dwThreadIds = new DWORD[m_ThreadsNo];

		bool allThreadsOK = true;

		// create the threads
		for(int i = 0; i &lt; m_ThreadsNo &amp;&amp; allThreadsOK; ++i)
		{
			hThreads[i] = ::CreateThread(
				NULL,
				0,
				threadProc,
				m_Params[i],
				startImmediatelly ? 0 : CREATE_SUSPENDED,
				&amp;dwThreadIds[i]);

			if(hThreads[i] == NULL)
			{
				for(int j = 0; j &lt; i; ++j)
				{
					::CloseHandle(hThreads[j]);
				}

				allThreadsOK = false;
			}
		}

		if(allThreadsOK)
		{
			// start the threads if they were suspended first
			if(!startImmediatelly)
			{
				for(int i = 0; i &lt; m_ThreadsNo; ++i)
				{
					::ResumeThread(hThreads[i]);
				}
			}

			// wait for all threads
			DWORD joinret = ::WaitForMultipleObjects(
				m_ThreadsNo,
				hThreads,
				TRUE,
				timeout);

			if(joinret == WAIT_FAILED)
			{

			}
			else if(joinret = WAIT_TIMEOUT)
			{

			}
			else if(joinret &gt;= WAIT_OBJECT_0 &amp;&amp; joinret &lt; WAIT_OBJECT_0 + m_ThreadsNo)
			{
				success = true;
			}
			else if(joinret &gt;= WAIT_ABANDONED_0 &amp;&amp; joinret &lt; WAIT_ABANDONED_0 + m_ThreadsNo)
			{

			}

			// close the thread handles
			for(int i = 0; i &lt; m_ThreadsNo; ++i)
			{
				::CloseHandle(hThreads[i]);
			}
		}

		delete [] hThreads;
		delete [] dwThreadIds;

		return success;
	}
};
</pre>
<p>This helper class contains:</p>
<ul>
<li>one parameter-less constructor that identifies the number of available processors and sets the threads count equal to the processors count</li>
<li>one constructor that takes the number of threads that should be created</li>
<li>one method (SetThreadParams) for setting the parameters for each thread that will be created</li>
<li>one method (Run) that creates and runs the thread, waits for them and checks the result of the execution</li>
</ul>
<p>As you can see the Run() method is simplistic. It does not handle for instance timed out or abandoned thread executions. Also it joins all threads, waiting until all of them finished execution. A more flexible method could wait only until the first thread finishes and then maybe closes the other threads. But as I said, this is a sample and not a complete solution.</p>
<p>Having this helper set up, I will start several threads to find the prime numbers in a sequence and print them in the console. </p>
<p>The following function computes whether a number is prime/</p>
<pre name="code" class="cpp">
#include &lt;cmath&gt;

bool IsPrime(int number)
{
	const int max = static_cast&lt; int &gt;(
		std::sqrt(static_cast&lt; double &gt;(number))) + 1;

	for (int i=2; i!=max; ++i)
	{
		if (number % i == 0) return false;
	}

	return true;
}
</pre>
<p>The thread procedure will run through a sub-sequence of a vector of integers and verify if each element is prime. I will use the following structure to pass the sequence bounds to the thread procedure:</p>
<pre name="code" class="cpp">
struct vector_bounds
{
	std::vector&lt; int &gt;::const_iterator begin;
	std::vector&lt; int &gt;::const_iterator end;
};
</pre>
<p>The thread procedure could look like this:</p>
<pre name="code" class="cpp">
static CRITICAL_SECTION cs;

DWORD WINAPI FindPrimes(LPVOID lpData)
{
	vector_bounds* bounds = static_cast&lt; vector_bounds* &gt;(lpData);
	if(bounds == NULL)
		return 1;

	for(std::vector&lt; int &gt;::const_iterator cit = bounds-&gt;begin;
		cit != bounds-&gt;end; ++cit)
	{
		if(IsPrime(*cit))
		{
			EnterCriticalSection(&amp;cs);

			std::cout &lt;&lt; *cit &lt;&lt; std::endl;

			LeaveCriticalSection(&amp;cs);
		}
	}

	return 0;
};
</pre>
<p>To print to the console a locking mechanism is necessary, otherwise prints from two different threads could collide. The critical section will be initialized before the threads are started.</p>
<p>What remains to be done is generating a sequence of integers, setting up the parameters with the sequence bounds for each thread and run the threads using the helper.</p>
<pre name="code" class="cpp">
int main()
{
	// generate some random numbers
	srand((unsigned long)time(NULL));
	std::vector&lt; int &gt; numbers;
	std::generate_n(std::back_inserter(numbers), 1000, rand);

	// create the thread helper
	ThreadHelper helper(4);
	int threads = helper.GetThreadsNo();

	// create the parameters for the threads
	std::vector&lt; vector_bounds &gt; params;
	std::vector&lt; int &gt;::const_iterator begin = numbers.begin();
	size_t partitionsize = numbers.size()/threads;

	for(int i = 0; i &lt; threads; ++i)
	{
		vector_bounds bound;
		bound.begin = begin;
		bound.end = (i == threads - 1) ? numbers.end() : begin + partitionsize;
		params.push_back(bound);

		begin = bound.end;
	}

	for(int i = 0; i &lt; threads; ++i)
		helper.SetThreadParams(i, &amp;params[i]);

	// run the threads
	InitializeCriticalSection(&amp;cs);

	std::cout &lt;&lt; "start running..." &lt;&lt; std::endl;

	bool success = helper.Run(FindPrimes, FALSE);

	std::cout &lt;&lt; "finished " &lt;&lt; (success? "successfully" : "failed") &lt;&lt; std::endl;

	DeleteCriticalSection(&amp;cs);

	return 0;
}
</pre>
<p>Having this threads helper class, what I need to do when running some processing in several threads is:</p>
<ul>
<li>setup thread parameters (if the case)</li>
<li>write the thread procedure</li>
<li>create a ThreadHelper object and initialize it</li>
<li>run the threads and collect the results</li>
</ul>
<p>The helper class prevents writing same code over and over again and helps focusing on the most important tasks: writing the thread procedure. As I said earlier it is not a full solution, nor one that fits all scenarios, but you can develop it to suit your needs.</p>
]]></content:encoded>
			<wfw:commentRss>http://mariusbancila.ro/blog/2010/02/01/helpers-for-multithreading-cpp/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
		<item>
		<title>STM.NET</title>
		<link>http://mariusbancila.ro/blog/2009/07/29/stm-net/</link>
		<comments>http://mariusbancila.ro/blog/2009/07/29/stm-net/#comments</comments>
		<pubDate>Wed, 29 Jul 2009 11:19:59 +0000</pubDate>
		<dc:creator>Marius Bancila</dc:creator>
				<category><![CDATA[.NET]]></category>
		<category><![CDATA[C#]]></category>
		<category><![CDATA[CTPs & Betas]]></category>
		<category><![CDATA[Parallel Programming]]></category>
		<category><![CDATA[Tools]]></category>
		<category><![CDATA[beta]]></category>
		<category><![CDATA[STM.NET]]></category>

		<guid isPermaLink="false">http://mariusbancila.ro/blog/?p=334</guid>
		<description><![CDATA[Microsoft has made available a first beta version of an experimental version of .NET 4.0, called .NET Framework 4.0 Beta 1 Enabled for Software Transactional Memory v1.0. Since that is quite a long name, the short one is STM.NET. This is a special version of .NET 4.0 that enables software transactional memory for C#. It [...]]]></description>
			<content:encoded><![CDATA[<p>Microsoft has made available a first beta version of an experimental version of .NET 4.0, called .NET Framework 4.0 Beta 1 Enabled for Software Transactional Memory v1.0. Since that is quite a long name, the short one is STM.NET. This is a special version of .NET 4.0 that enables software transactional memory for C#. It allows programmers to demarcate regions of code as operating in an atomic, isolated transaction from other code running concurrently. The means to do this is a delegate called Atomic.Do, or try-catch blocks. Might be that in the future an &#8216;atomic&#8217; block will be added to the language(s).</p>
<p>This first version of the framework, also comes with additional tools:</p>
<ul>
<li>tooling (debugging, ETW tracing)</li>
<li>lock interoperability</li>
<li>interoperability with traditional transactions</li>
<li>annotations (how methods run in transactions, suppressed transactions on methods, etc.)</li>
<li>static and dynamic checking of annotations</li>
</ul>
<p>On the other hand there are some limitations:</p>
<ul>
<li>only works for C# for now</li>
<li>cannot be installed on a machine with VS 2010, nor the opposite</li>
<li>there is only a 32-bit version</li>
</ul>
<p>More information about it can be found at the <a href="http://blogs.msdn.com/stmteam/">STM team blog</a> or <a href="http://msdn.microsoft.com/en-us/devlabs/ee334183.aspx">MSDN DevLabs</a>.</p>
]]></content:encoded>
			<wfw:commentRss>http://mariusbancila.ro/blog/2009/07/29/stm-net/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Axum, A Language for Building Parallel Applications</title>
		<link>http://mariusbancila.ro/blog/2009/05/13/axum-a-language-for-building-parallel-applications/</link>
		<comments>http://mariusbancila.ro/blog/2009/05/13/axum-a-language-for-building-parallel-applications/#comments</comments>
		<pubDate>Wed, 13 May 2009 06:26:09 +0000</pubDate>
		<dc:creator>Marius Bancila</dc:creator>
				<category><![CDATA[.NET]]></category>
		<category><![CDATA[CTPs & Betas]]></category>
		<category><![CDATA[Parallel Programming]]></category>
		<category><![CDATA[agents]]></category>
		<category><![CDATA[axum]]></category>
		<category><![CDATA[isolation]]></category>
		<category><![CDATA[parallel]]></category>

		<guid isPermaLink="false">http://mariusbancila.ro/blog/?p=268</guid>
		<description><![CDATA[Last week Microsoft published on DevLabs a .NET language for building parallel applications, called Axum, and earlier known as Maestro. This new language is build on the architecture of the web, on the principles of isolation, message-passing, fault-tolerance, loose-coupling. It is said to have a more succinct syntax than Erlang, and have the isolation advantage [...]]]></description>
			<content:encoded><![CDATA[<p>Last week Microsoft published on <a href="http://msdn.microsoft.com/en-us/devlabs/dd795202.aspx">DevLabs</a> a .NET language for building parallel applications, called Axum, and earlier known as Maestro. This new language is build on the architecture of the web, on the principles of isolation, message-passing, fault-tolerance, loose-coupling. It is said to have a more succinct syntax than Erlang, and have the isolation advantage over MPI, CCR and Asynchronous Agents.</p>
<p>Isolation is key in this architecture and is achieved with:</p>
<ul>
<li>domains, that limits the runtime scope of data to its compile-time scope (objects don&#8217;t escape domains)</li>
<li>agents, active components that provide access to domains, and live in a thread of their own, different that the callers; their methods are not accessible outside;</li>
<li>channels, are the mean to communicate with agents; they are established by the runtime, when agents are created. The most important parts of the channels are the ports (input or output), that can be viewed as queues in which data is placed.</li>
</ul>
<p>Here are more readings about these topics here:</p>
<ul>
<li><a href="http://blogs.msdn.com/maestroteam/archive/2009/02/27/we-haven-t-forgotten-about-other-models-honest.aspx">Axum core architecture principles</a></li>
<li><a href="http://blogs.msdn.com/maestroteam/archive/2009/02/27/isolation-in-maestro.aspx">Isolation in Axum</a></li>
<li><a href="http://blogs.msdn.com/maestroteam/archive/2009/03/02/channels.aspx">Channels</a></li>
</ul>
<p>You can find more about Axum at:</p>
<ul>
<li><a href="http://msdn.microsoft.com/en-us/devlabs/dd795202.aspx">Home page at Microsoft DevLabs</a></li>
<li><a href="http://blogs.msdn.com/maestroteam/">Axum Team Blog</a></li>
<li><a href="http://social.msdn.microsoft.com/Forums/en-US/axum/threads">Axum Forum</a></li>
</ul>
<p></p>
]]></content:encoded>
			<wfw:commentRss>http://mariusbancila.ro/blog/2009/05/13/axum-a-language-for-building-parallel-applications/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Word Reducing Puzzle</title>
		<link>http://mariusbancila.ro/blog/2008/06/05/word-reducing-puzzle/</link>
		<comments>http://mariusbancila.ro/blog/2008/06/05/word-reducing-puzzle/#comments</comments>
		<pubDate>Thu, 05 Jun 2008 11:52:35 +0000</pubDate>
		<dc:creator>Marius Bancila</dc:creator>
				<category><![CDATA[F#]]></category>
		<category><![CDATA[Parallel Programming]]></category>

		<guid isPermaLink="false">http://mariusbancila.ro/blog/?p=124</guid>
		<description><![CDATA[I recently found an interesting problem on the web, about reducing words, letter by letter until only one letter remains. Here is a formal definition: We define word reduction as removing a single letter from a word while leaving the remaining letters in their original order so that the resulting sequence of characters is also [...]]]></description>
			<content:encoded><![CDATA[<p>I recently found an interesting problem on the web, about reducing words, letter by letter until only one letter remains. Here is a formal definition:</p>
<blockquote><p>
We define word reduction as removing a single letter from a word while leaving the remaining letters in their original order so that the resulting sequence of characters is also a word. A good word can be reduced step-by-step until all that is left is a or i. Your program will answer the question: what is the biggest word in the given dictionary that can be reduced?
</p></blockquote>
<p>A typical example, not very long is this:</p>
<pre class="prettyprint">
planets
plants
pants
pant
ant
an
a
</pre>
<p>So I though this could be a good exercise for F#. Several good dictionaries, both small and big, can be found <a href="http://cs.millersville.edu/~katz/cs362/examples/dictionaries/" target="_blank">here</a>.</p>
<p>My approach was to read the dictionary and build a list for each word length: one for 1-letter words, one for 2-letter words, etc. This could be a Dictionary&lt;int, list<string>&gt;, and let&#8217;s call it simply dictionary. Then these lists could be traversed and create a second set of lists (let&#8217;s call this reducedwords), but only with the words that meet the defined reduction.</p>
<p>An good approach would be to take each list of words from the initial dictionary, starting with the list with words of 2 letters, and for each word to delete one letter at a time. Then check to see if the resulting word already exists in the list from reducedwords corresponding to a length smaller with 1. If it&#8217;s there, that this word could be reduced and should be added to the list from reducedwords corresponding to the current length. In other words, the algorithm would be:</p>
<pre class="prettyprint">
copy dictionary[1] to reducedwords[1]

for length = 2 to maxwordlength do
  foreach word in dictionary[length]
    foreach letter in word
      * delete the letter
      * if new word exists in reducedwords[length-1] then
          * add new word to reducedwords[length]
</pre>
<p>Here is a function for reading a dictionary file:</p>
<pre class="prettyprint">
let readWords (filename:string) =
   let dictionary = new Dictionary< int, list< string > >()
   let reader = new StreamReader(filename)
   let word = String.Empty
   let fileend = ref false
   while (!fileend = false) do
      let word = reader.ReadLine()
      if word = null then
         fileend := true
      else
         let len = String.length word
         let ok, words = dictionary.TryGetValue(len)
         if ok then dictionary.[len] <- words@[word]
         else dictionary.[len] <- [word]
   done
   dictionary.[1] <- ["a";"e";"i";"o";"u"]
   dictionary
</pre>
<p>One I have the dictionary read, I can apply the algorithm and generate a second Dictionary structure. The following function also returns length of the longest word(s) in the reduced dictionary. This is useful for printing.</p>
<pre class="prettyprint">
let findReducedWords (dictionary:Dictionary< int, list< string > >) =
   let reducedwords = new Dictionary< int, list< string > >()
   reducedwords.[1] <- dictionary.[1]

   let notdone = ref true
   let i = ref 2
   while (!notdone = true) do
      let ok, words = dictionary.TryGetValue(!i)
      if ok <> true then
         notdone := false
      else
         let added = ref false
         let ok, reducedpre = reducedwords.TryGetValue(!i - 1)
         reducedwords.[!i] <- []
         words |> List.iter (fun word ->
            for j = 0 to word.Length-1 do
               let trimmedword = word.Remove(j, 1)
               if reducedpre.Exists(fun x -> x = trimmedword) then
                  reducedwords.[!i] <- reducedwords.[!i]@[word]
                  added := true
            done;
         )
         reducedwords.[!i] <- reducedwords.[!i] |> Set.of_list |> List.of_seq
         if !added then
            i := !i + 1
         else
            reducedwords.Remove(!i) |> ignore
            notdone := false
   done

   (reducedwords, !i-1)
</pre>
<p>Since the problem is about printing only the longest such reductions, I'll only consider starting from the list of reduced words that has the longest words. That's why I needed findReducedWords to return the length of longest reducible word.</p>
<p>To print these paths I apply the same algorithm as before. The only difference is that I build a list with the words in the reducing path, starting with the longest word and ending with a 1-letter word.</p>
<pre class="prettyprint">
let rec printSequence
   (word:string)
   (reducedwords:Dictionary< int, list< string > >)
   (path:list< string >) =
   match word.Length with
   | 1 ->
      path@[word] |> List.iter (fun x -> printf "%s " x);
      printfn ""
   | _ ->
      let ok, reducedpre = reducedwords.TryGetValue(word.Length-1)
      if ok then
         for j = 0 to word.Length-1 do
            let trimmedword = word.Remove(j, 1)
            if reducedpre.Exists(fun x -> x = trimmedword) then
               printSequence trimmedword reducedwords (path@[word])
         done;

let printSequences
   (reducedwords:Dictionary< int, list< string > >)
   (maxlen:int) =
   let ok, words = reducedwords.TryGetValue(maxlen)
   if ok then
      words |> List.iter (fun word ->
         printSequence word reducedwords [])
</pre>
<p>The only thing left to do is calling all these functions:</p>
<pre class="prettyprint">
let main()=
   printfn "reading dictionary..."
   let dictionary = readWords "huge_dict.txt"

   printfn "building structures..."
   let reducedwords, max = findReducedWords dictionary

   printfn "printing matches..."
   printSequences reducedwords max

   Console.WriteLine("Press any key to continue...")
   Console.ReadKey()

main()
</pre>
<p>My results for <a href="http://cs.millersville.edu/~katz/cs362/examples/dictionaries/huge.dict" target="_blank">this dictionary</a> were:</p>
<pre class="prettyprint">
complecting completing competing compting comping coping oping ping pig pi i
complecting completing competing compting comping coping oping ping pin in i
complecting completing competing compting comping coping oping ping pin pi i
</pre>
<p></p>
]]></content:encoded>
			<wfw:commentRss>http://mariusbancila.ro/blog/2008/06/05/word-reducing-puzzle/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
		<item>
		<title>Applying File Transformations with F#</title>
		<link>http://mariusbancila.ro/blog/2008/05/06/applying-file-transformations-with-f/</link>
		<comments>http://mariusbancila.ro/blog/2008/05/06/applying-file-transformations-with-f/#comments</comments>
		<pubDate>Tue, 06 May 2008 20:10:57 +0000</pubDate>
		<dc:creator>Marius Bancila</dc:creator>
				<category><![CDATA[F#]]></category>
		<category><![CDATA[Parallel Programming]]></category>
		<category><![CDATA[files]]></category>
		<category><![CDATA[parallel]]></category>
		<category><![CDATA[pattern]]></category>
		<category><![CDATA[record]]></category>
		<category><![CDATA[recursive]]></category>
		<category><![CDATA[regex]]></category>

		<guid isPermaLink="false">http://mariusbancila.ro/blog/?p=121</guid>
		<description><![CDATA[In this post I&#8217;ll show some F# constructs, all put together in a simple application that modifies file names that match a criteria. This would be an application that is started from a console with the following command line options: filesmod.exe -f < folder > [-r] -p < pattern > [-pre < prefix >] [-suf [...]]]></description>
			<content:encoded><![CDATA[<p>In this post I&#8217;ll show some F# constructs, all put together in a simple application that modifies file names that match a criteria. This would be an application that is started from a console with the following command line options:</p>
<pre class="console">
filesmod.exe -f < folder > [-r] -p < pattern > [-pre < prefix >] [-suf < suffix >]
  -f < folder>   specifies the folder where the files are located
  -r            indicates that the specified folder should be parsed
                recursively
  -p < pattern>  indicates a pattern used for filtering files
  -pre < prefix> indicats a prefix to the added to all files
                that match the criteria
  -suf < suffix> indicats a suffix to the added to all files
                that match the criteria
</pre>
<h3>Reading command line</h3>
<p>The command line arguments can be retrieved using the Environment class from the .NET framework. This class has a static method called GetCommandLineArgs() that returns a list of the passed arguments.<br />
We can define a type that contains all the parsed arguments.</p>
<pre class="prettyprint">
type CommandOptions =
    { mutable Folder : string;
      mutable Recursive : bool;
      mutable Pattern : string;
      mutable Prefix : string;
      mutable Suffix : string;}
</pre>
<p>This mutable record can be instantiated, and the value can be mutated while parsing the arguments. This is how you instantiate it:</p>
<pre class="prettyprint">
   let cmdops =
      { new CommandOptions
        with Folder = String.Empty
        and Recursive = false
        and Pattern = String.Empty
        and Prefix = String.Empty
        and Suffix = String.Empty }
</pre>
<p>Parsing the command line arguments can be done with pattern matching. This is the equivalent of switches in C+/C#/Java, only more powerful.<br />
Basically, I&#8217;m checking each argument, and if it&#8217;s a flag in the command line (-f, -r, -p, -pre, -sub) I take the next argument and put it in the appropriate property of the record.</p>
<pre class="prettyprint">
   try
      let args = Environment.GetCommandLineArgs()
      for i = 0 to args.Length-1 do
         match args.(i) with
            | "-f" when i+1 <= args.Length-1 -> cmdops.Folder <- args.(i+1)
            | "-r" -> cmdops.Recursive <- true
            | "-p" when i+1 <= args.Length-1 -> cmdops.Pattern <- args.(i+1)
            | "-pre" when i+1 <= args.Length-1 -> cmdops.Prefix <- args.(i+1)
            | "-suf" when i+1 <= args.Length-1 -> cmdops.Suffix <- args.(i+1)
            | _ -> ()
      done
   with e -> printfn "%s" e.Message
</pre>
<p>There are two things you could notice here. The first is the try &#8230; with block that makes sure any possible exception is caught.<br />
The second is the quarding the rules with the contidion that the current argument is not the last one in the list. (-f should be followed by a folder, -suf by a suffix, etc.)<br />
You can see what in the when statement:</p>
<pre class="prettyprint">
when i+1 <= args.Length-1
</pre>
<h3>Getting the files in a directory</h3>
<p>We can get all files in a folder, using the following algorithm:</p>
<ul>
<li>get all the files in the current folder</li>
<li>get all the sub-folders in the current folder and for each of them apply the algorithm again</li>
</ul>
<p>That is spelled "recursion"!. Our function should take several arguments: the path of a folder, a pattern for mathing filenames and a flag indicating whether sub-folders should be parsed or not.</p>
<pre class="prettyprint">
let rec allFiles dir pattern r =
    seq
        { for file in Directory.GetFiles(dir, pattern) do
            yield file
          if r then
            for subdir in Directory.GetDirectories(dir) do
                for file in allFiles subdir pattern r do
                    yield file }
</pre>
<p>The above function is recursive and returns a sequence of file names. Sequences are lazy, which means that successive elements are computed and returned on demand, when they are needed.<br />
That is the opposite of a list or array, whose elements are created at once. The keyword 'yield' here is used to return a new value as the sequence is iterated.</p>
<h3>Processing the files</h3>
<p>To process the files, we simply iterate over the sequence of files from the specified folder, match it against the provided parttern, and if there is a match, apply the prefix and/or suffix transformation.</p>
<pre class="prettyprint">
   for name in (allFiles cmdops.Folder "*.*" cmdops.Recursive) do
      let file = new FileInfo(name)
      if(Regex.IsMatch(file.Name, cmdops.Pattern, RegexOptions.Singleline)) then
        let filename = file.Name.Substring(0, file.Name.LastIndexOf('.'))
        let newname = file.Directory.FullName+"\\"+cmdops.Prefix+filename+cmdops.Suffix+file.Extension
        System.IO.File.Move(file.FullName, newname)
        printfn "%s -> %s" file.FullName newname
   done
</pre>
<p>Well, I have two cores on my machine, and since the Parallel FX framework is available, I like to use it. So here is the parallel version of that:</p>
<pre class="prettyprint">
       try
          Parallel.ForEach(allFiles cmdops.Folder "*.*" cmdops.Recursive, fun name ->
             let file = new FileInfo(name)
             if(Regex.IsMatch(file.Name, cmdops.Pattern, RegexOptions.Singleline)) then
                let filename = file.Name.Substring(0, file.Name.LastIndexOf('.'))
                let newname = file.Directory.FullName+"\\"+cmdops.Prefix+filename+cmdops.Suffix+file.Extension
                System.IO.File.Move(file.FullName, newname)
                printfn "%s -> %s" file.FullName newname)
       with e -> printfn "%s" e.InnerException.Message
</pre>
<p>The provided (via command line) pattern is a regular expression. Initially, the folder is checked for all files and then these files are matched against this regular expression.</p>
<p>As I was saying in a previous post, if you use PFX, you have to add a reference to the System.threading.dll assembly, which requires a reference to the System.Core.dll assembly.<br />
That should be specified at the project's propertyes.</p>
<blockquote><p>
-r C:\WINDOWS\assembly\GAC_MSIL\System.Core\3.5.0.0__b77a5c561934e089\System.Core.dll -r "C:\Program Files\Microsoft Parallel Extensions Dec07 CTP\System.Threading.dll"
</p></blockquote>
<h3>Putting all together</h3>
<p>All that put together looks like this:</p>
<pre class="prettyprint">
#light

open System
open System.IO
open System.Text.RegularExpressions

open System.Threading

let rec allFiles dir pattern r =
    seq
        { for file in Directory.GetFiles(dir, pattern) do
            yield file
          if r then
            for subdir in Directory.GetDirectories(dir) do
                for file in allFiles subdir pattern r do
                    yield file }

let showUsage() =
    printfn "filesmod.exe -f < folder > [-r] -p < pattern > [-pre < prefix >] [-suf < suffix >]"
    printfn "  -f < folder >\tspecifies the folder where the files are located"
    printfn "  -r\t\tindicates that the specified folder should be parsed\n\t\trecursively"
    printfn "  -p < pattern >\tindicates a pattern used for filtering files"
    printfn "  -pre < prefix >\tindicats a prefix to the added to all files\n\t\tthat match the criteria"
    printfn "  -suf < suffix >\tindicats a suffix to the added to all files\n\t\tthat match the criteria"

type CommandOptions =
    { mutable Folder : string;
      mutable Recursive : bool;
      mutable Pattern : string;
      mutable Prefix : string;
      mutable Suffix : string;}

let main()=
   let cmdops =
      { new CommandOptions
        with Folder = String.Empty
        and Recursive = false
        and Pattern = String.Empty
        and Prefix = String.Empty
        and Suffix = String.Empty }

   try
      let args = Environment.GetCommandLineArgs()
      for i = 0 to args.Length-1 do
         match args.(i) with
            | "-f" when i+1 <= args.Length-1 -> cmdops.Folder <- args.(i+1)
            | "-r" -> cmdops.Recursive <- true
            | "-p" when i+1 <= args.Length-1 -> cmdops.Pattern <- args.(i+1)
            | "-pre" when i+1 <= args.Length-1 -> cmdops.Prefix <- args.(i+1)
            | "-suf" when i+1 <= args.Length-1 -> cmdops.Suffix <- args.(i+1)
            | _ -> ()
      done
   with e -> printfn "%s" e.Message

   if ((String.IsNullOrEmpty(cmdops.Prefix) &#038;&#038; String.IsNullOrEmpty(cmdops.Suffix)) ||
        String.IsNullOrEmpty(cmdops.Pattern) ||
        String.IsNullOrEmpty(cmdops.Folder)) then
        showUsage()
   else
       try
          Parallel.ForEach(allFiles cmdops.Folder "*.*" cmdops.Recursive, fun name ->
             let file = new FileInfo(name)
             if(Regex.IsMatch(file.Name, cmdops.Pattern, RegexOptions.Singleline)) then
                let filename = file.Name.Substring(0, file.Name.LastIndexOf('.'))
                let newname = file.Directory.FullName+"\\"+cmdops.Prefix+filename+cmdops.Suffix+file.Extension
                System.IO.File.Move(file.FullName, newname)
                printfn "%s -> %s" file.FullName newname)
       with e -> printfn "%s" e.InnerException.Message

   Console.WriteLine("Press any key to continue...")
   Console.ReadKey()

main()
</pre>
<p>Of course, the options available in this application (on file name changes) are pretty limited, but that can be extended at will.</p>
]]></content:encoded>
			<wfw:commentRss>http://mariusbancila.ro/blog/2008/05/06/applying-file-transformations-with-f/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
	</channel>
</rss>

