<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>KWrite &#187; Programming</title>
	<atom:link href="http://blog.kentran.net/category/computer/programming/feed/" rel="self" type="application/rss+xml" />
	<link>http://blog.kentran.net</link>
	<description>Get Real, Be Rational</description>
	<lastBuildDate>Sun, 05 Sep 2010 18:18:09 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.0.1</generator>
		<item>
		<title>&#8220;Bibles&#8221; in High Performance Computing</title>
		<link>http://blog.kentran.net/2010/03/bibles-in-high-performance-computing/</link>
		<comments>http://blog.kentran.net/2010/03/bibles-in-high-performance-computing/#comments</comments>
		<pubDate>Sun, 14 Mar 2010 03:18:07 +0000</pubDate>
		<dc:creator>Kenneth Tran</dc:creator>
				<category><![CDATA[Math]]></category>
		<category><![CDATA[Programming]]></category>
		<category><![CDATA[Sciences]]></category>

		<guid isPermaLink="false">http://blog.kentran.net/?p=134</guid>
		<description><![CDATA[<p>This is the continuation of my series &#8220;Bibles&#8221; in Applied Math. I write a new page for this section because it is a bit off from what most people call &#8220;Applied Math&#8221;. Further, this topic deserves a blog entry, or even a forum, on its own because being able to make your code run <span style="color:#777"> . . . &#8594; Read More: <a href="http://blog.kentran.net/2010/03/bibles-in-high-performance-computing/">&#8220;Bibles&#8221; in High Performance Computing</a></span>]]></description>
			<content:encoded><![CDATA[<p>This is the continuation of my series <a href="http://blog.kentran.net/2008/03/30/bible/">&#8220;Bibles&#8221; in Applied Math</a>. I write a new page for this section because it is a bit off from what most people call &#8220;Applied Math&#8221;. Further, this topic deserves a blog entry, or even a forum, on its own because being able to make your code run 10-15% faster (or even 90% faster if you haven&#8217;t mastered certain level of the art of programming) is a big deal nowadays. It can even land you a lucrative job (<a href="http://www.nytimes.com/2009/07/24/business/24trading.html?_r=3&amp;partner=rss&amp;emc=rss">here&#8217;s an example</a>).</p>
<p>Suppose that you&#8217;re a master of Numerical Algorithms. You&#8217;ve spent 2 years working on a complex engineering problem and recently have developed a new algorithm that can solve the problem in linear time instead of the state-of-the-art <img src='http://s.wordpress.com/latex.php?latex=O%5Cleft%28N%5Clog%20N%29%5Cright%29&#038;bg=ffffff&#038;fg=000000&#038;s=0' alt='O\left(N\log N)\right)' title='O\left(N\log N)\right)' class='latex' />. Your new solver (and the 2 years) would be wasted if it is poorly written and/or poorly compiled. Here are a couple out of thousands of common poor programming practices.</p>
<p><strong>1. Computing the same thing over and over again</strong></p>
<pre class="brush: cpp;">
for (int k=0;k&lt;100;k++) {
 out[k] = 0;
 for (int i=0;i&lt;M;i++)
  for (int j=0;j&lt;N;j++)
   out[k] += kernel(i,j) * in[k];
}
</pre>
<p>In this example, kernel(i,j) is re-computed 99 times. Because this matrix Kernel is independent from the outer loop, it can and most of the times should be pre-computed. If the computation of kernel is computationally intensive, which is common, pre-computation of the matrix can make your program run 99.9 times faster.</p>
<p><strong>2. Using the right flags when compiling</strong></p>
<pre class="brush: plain;">
[zer0ne@ion]$ g++ myProgram.cpp -o myProgram
[zer0ne@ion]$ ./myProgram
Total time: 68.74 seconds
[zer0ne@ion]$ g++ -O3 myProgram.cpp -o myProgram
[zer0ne@ion]$ ./myProgram
Total time: 13.9 seconds
[zer0ne@ion]$ g++ -O3 -DNDEBUG myProgram.cpp -o myProgram
[zer0ne@ion]$ ./myProgram
Total time: 11.54 seconds
</pre>
<p>So, if you&#8217;re already good at designing algorithms, it&#8217;s worth to learn a bit of performance optimization tricks. I am no expert in HPC, let alone related fields such as Computer  Architectures or Compilers, so my best bet is to point you to some great books out there. Also for the same reason, I&#8217;ll appreciate if you share your own tips.</p>
<ol>
<li><a href="http://www.amazon.com/Performance-Optimization-Numerically-Intensive-Environments/dp/0898714842/ref=sr_1_1?ie=UTF8&amp;s=books&amp;qid=1268530349&amp;sr=8-1">Performance Optimization of Numerically Intensive Codes by Hoisie</a>. $73 seems high for a 173-page book but this thin bible is worth every penny. It covers all basic stuffs such as CPU architecture, compiler optimization, memory localit, profiling, etc.</li>
<li><a href="http://www.amazon.com/Performance-Computing-Architectures-Optimization-Benchmarks/dp/156592312X/ref=sr_1_1?ie=UTF8&amp;s=books&amp;qid=1268536364&amp;sr=1-1">High Performance Computing by Kevin Dowd &amp; Charles Severance</a>. Similar contents but this book explains things in greater depth.</li>
</ol>
]]></content:encoded>
			<wfw:commentRss>http://blog.kentran.net/2010/03/bibles-in-high-performance-computing/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Programming Tips (C++)</title>
		<link>http://blog.kentran.net/2008/03/programming-tips-c/</link>
		<comments>http://blog.kentran.net/2008/03/programming-tips-c/#comments</comments>
		<pubDate>Sun, 02 Mar 2008 17:03:51 +0000</pubDate>
		<dc:creator>Kenneth Tran</dc:creator>
				<category><![CDATA[Programming]]></category>
		<category><![CDATA[Technology]]></category>
		<category><![CDATA[C++]]></category>

		<guid isPermaLink="false">http://www.khoatran.com/blog/?p=52</guid>
		<description><![CDATA[<p>Since I&#8217;ve been back coding for the last few days, I&#8217;d like to save some useful programming tips that I&#8217;ve just figured out.</p> <p>1. Linear Algebra package: use uBlas. uBlas, developped by Boost, is very object-oriented and easy to use. Not only does it support linear algebra, it also nicely supports sparse matrix storage <span style="color:#777"> . . . &#8594; Read More: <a href="http://blog.kentran.net/2008/03/programming-tips-c/">Programming Tips (C++)</a></span>]]></description>
			<content:encoded><![CDATA[<p>Since I&#8217;ve been back coding for the last few days, I&#8217;d like to save some useful programming tips that I&#8217;ve just figured out.</p>
<p><strong>1. Linear Algebra package</strong>: use <a href="http://www.boost.org/libs/numeric/ublas/doc/index.htm" target="_blank">uBlas</a>. uBlas, developped by <a href="http://en.wikipedia.org/wiki/Boost_C%2B%2B_Libraries" target="_blank">Boost</a>, is very object-oriented and easy to use. Not only does it support linear algebra, it also nicely supports sparse matrix storage and processing, which is important for iterative methods (when solving PDEs by Finite Difference or Finite Element). Take a look at my little program below, which constructs a sparse matrix when solving a boundary value problem.<br />
<!-- BEGIN TEMPLATE: bbcode_code --></p>
<p class="smallfont" style="margin-bottom: 2px">Code:</p>
<pre style="border: 1px inset ; margin: 0pt; padding: 2px; overflow: auto; width: 500px; height: 466px; text-align: left">#include &lt;boost/numeric/ublas/vector.hpp&gt;

#include &lt;boost/numeric/ublas/matrix_sparse.hpp&gt;

using namespace std;

namespace ublas = boost::numeric::ublas;const int N = 5000; // number of discretizations on each direction

typedef ublas::vector&lt;double&gt; Vector;

typedef ublas::compressed_matrix&lt;double&gt; Matrix; // Sparse Matrix class

void initializeMatrix(Matrix&amp; A) {

    long k=0;

    for (int i=0; i&lt;N; i++)

        for (int j=0; j&lt;N; j++) {

            A(k,k) = -4;

            // adjacent elements in x-direction

            if (i&gt;0) A(k,k-N)=1;

            if (i&lt;N-1)

                if (i==0) A(k,k+N)=2; // Neumann condition on this side

                else A(k,k+N)=1; //Dirichlet condition on this side

            // adjacent elements in y-direction

            if (j&gt;0) A(k,k-1)=1;

            if (j&lt;N-1)

                if (j==0) A(k,k+1)=2; // Neumann condition

                else A(k,k+1)=1; // Dirichlet condition

k++;

        }

}</pre>
<p><!-- END TEMPLATE: bbcode_code -->Drawbacks of uBlas</p>
<ul>
<li>It has poor performance compared to (atlas-)Blas. One reason is that it is the most abstract linear algebra library (easier to use). Another critical reason is that it performs many *useful* (debugging) checks. The performance issue can be overcome through some optimization tricks, such as one that I described in the <strong>compiler</strong> section. Check out the <a href="http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl?Effective_UBLAS" target="_blank">effective uBlas wiki</a> page for more performance tips. If performance is critical, consider <a href="http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl?Linear_Algebra_With_UBLAS" target="_blank">binding with Atlas</a>.</li>
<li>Poor documentation: if you&#8217;re used to Java&#8217;s API (which is terrific) as I was, using uBlas as first may be difficult because uBlas&#8217;s documentation is, now, quite short and lacks many details. The developers are working hard on it though. Moreover, they also maintain a main mailing list (and some other satellite sites) where you can ask questions.</li>
</ul>
<p><strong>2. Quant Finance Library: </strong>use <a href="http://quantlib.org/reference/index.html" target="_blank"><strong>QuantLib</strong></a>, the major open-source library</p>
<p><strong>3. Compiler</strong>: when finished with your programming/debugging, consider using <strong>-O3</strong> and <strong>-DNDEBUG</strong> for a much better performance (thanks to Tuyen). Most of the time, it optimizes the program significantly. Below is an illustration.<br />
<!-- BEGIN TEMPLATE: bbcode_code --></p>
<p class="smallfont" style="margin-bottom: 2px">Code:</p>
<pre style="border: 1px inset ; margin: 0pt; padding: 2px; overflow: auto; width: 500px; height: 210px; text-align: left">[khoa@trinidad]$ g++ myProg.cpp -o myProg

[khoa@trinidad]$ ./myProg

Total time: <strong>68.74</strong> seconds

[khoa@trinidad]$ g++ <strong>-DNDEBUG</strong> myProg.cpp -o myProg

[khoa@trinidad]$ ./myProg

Total time: <strong>49.43</strong> seconds

[khoa@trinidad courses]$ g++ <strong>-O3</strong> myProg.cpp -o myProg

[khoa@trinidad courses]$ ./myProg

Total time: <strong>13.9</strong> seconds

[khoa@trinidad]$ g++ <strong>-O3 -DNDEBUG</strong> myProg.cpp -o myProg

[khoa@trinidad]$ ./myProg

Total time: <strong>11.54</strong> seconds</pre>
]]></content:encoded>
			<wfw:commentRss>http://blog.kentran.net/2008/03/programming-tips-c/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
	</channel>
</rss>
