Changeset - 787f58c121ff
[Not reviewed]
0 1 0
Bradley Kuhn (bkuhn) - 8 years ago 2016-08-09 17:25:51
bkuhn@ebb.org
Fix incorrect word.
1 file changed with 1 insertions and 1 deletions:
0 comments (0 inline, 0 general)
www/conservancy/static/copyleft-compliance/vmware-code-similarity.html
Show inline comments
...
 
@@ -39,25 +39,25 @@ document.write('<a h'+'ref'+'="ma'+'ilto'+':'+e+'">'+e+'<\/'+'a'+'>');
 
</script><noscript>&#116;&#x6f;&#114;&#118;&#x61;&#108;&#100;&#x73;&#32;&#x61;&#116;&#32;&#108;&#x69;&#110;&#x75;&#120;&#x2d;&#102;&#x6f;&#x75;&#110;&#100;&#x61;&#116;&#x69;&#x6f;&#110;&#32;&#100;&#x6f;&#116;&#32;&#x6f;&#114;&#x67;</noscript> on 2010-05-16 14:17:36 -0700, with Git commit comment: &quot;Linus 2.6.34&quot;.</p>
 
<h2 id="comparing-hellwigs-contributions-from-linux-historical-repository-to-vmware-sources">Comparing Hellwig's Contributions From Linux Historical Repository to VMware Sources</h2>
 
<p>I then used this corpus as input to CCFinderX (similar to the other CCFinderX comparisons explained earlier). Specifically, this CCFinderX comparison compared all known Hellwig-contributed material from the historical Linux repository to the partial VMware source release. CCFinderX found a ratio of similarity of 0.0900% between Hellwig's code and the source code in VMware's (partial) source release. CCFinderX specifically identified 12 distinct locations where substantial sections of code contributed by Hellwig appeared in VMware's code.</p>
 
<p>Most notably, substantial portions of the the following core SCSI functions were found by this search technique: <code>__scsi_device_lookup</code> and <code>__scsi_get_command</code>, <code>mpt_get_product_name</code>, <code>scsi_proc_host_rm</code>, <code>mega_enum_raid_scsi</code>, <code>mega_m_to_n</code>, <code>mega_prepare_passthru</code>, <code>proc_scsi_show</code>, and <code>__down_read_trylock</code>.</p>
 
<h2 id="extracting-hellwigs-contributions-from-modern-linux-repository">Extracting Hellwig's Contributions From Modern Linux Repository</h2>
 
<p>Beginning on 2005-04-16, Linux began using the new Git system to store revision history. This history can be analyzed in a similar fashion as was done for the historical repository.</p>
 
<p>In this case, I picked a specific revision to center the analysis, the Linux 2.6.34 release from 2010-05-16. For the period from 2005-04-16 through 2010-05-16, I extracted from the modern Linux tree the identifying number of all commits that are either made with Hellwig in the official Author field, or where the person in the Author field left notes clearly indicating that the contribution was done by Hellwig. For the latter, the following regular expression search against the log file was used (as before):</p>
 
<pre><code>(Submitted\s+by|original\s+patch|patch\s+(from|by)|originally\s+(from|by)).*Hellwig</code></pre>
 
<p>Specifically, I used the <a href="https://github.com/conservancy/gpl-compliance-tools/blob/master/commit-id-list-matching-regex.plx">same script as before</a> to now extract a list of commit ids from the <a href="git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git">modern Linux repository</a>. This method found 2,595 separate occasions of contribution by Hellwig from 2005-04-16 through 2010-05-16.</p>
 
<p>As before, after finding these separate occasions of contribution, I then extracted the source code lines that Hellwig added or changed in each contribution in this repository. I did so by carefully cross-referencing the commits that Hellwig performed with the output of <code>git blame</code>. I specifically <a href="https://github.com/conservancy/gpl-compliance-tools/blob/master/extract-code-added-in-commits.plx">used the same script as before</a> to carefully extracted only lines that Hellwig changed or added in that repository, and placed only those contributions identifiable as Hellwig's into new files whose named matched the original filenames. This created a corpus of code that can be verifiable as added or changed by Hellwig and no one else.</p>
 
<h2 id="comparing-hellwigs-contributions-from-modern-linux-repository-to-vmware-sources">Comparing Hellwig's Contributions From Modern Linux Repository to VMware Sources</h2>
 
<p>I then used this corpus as input to CCFinderX again. Specifically, this CCFinderX comparison compared all known Hellwig-contributed material from the modern Linux repository to the partial VMware source release. CCFinderX found a ratio of similarity of 0.1615% between Hellwig's code and the source code in VMware's (partial) source release was contributed by Hellwig. CCFinderX specifically identified 23 distinct locations where substantial sections of code contributed by Hellwig appeared. These 23 locations are found in the following 19 functions: <code>mptsas_init</code>, <code>mptsas_get_linkerrors</code>, <code>megasas_build_and_issue_cmd</code>, <code>cciss_getgeo</code>, <code>mptsas_get_bay_identifier</code>, <code>phy_to_ioc</code>, <code>mptsas_sas_enclosure_pg0</code>, <code>SendIocInit</code>, <code>mptsas_parse_device_info</code>, <code>csmisas_sas_device_pg0</code>, <code>mptsas_sas_io_unit_pg0</code>, <code>mptsas_sas_io_unit_pg1</code>, <code>mptsas_sas_expander_pg1</code>, <code>mptsas_sas_enclosure_pg0</code>, <code>aac_handle_aif</code>, <code>mptsas_get_bay_identifier</code>, <code>mpt_host_page_alloc</code>, <code>mptsas_probe_one_phy</code>.</p>
 
<h2 id="changed-and-added-lines-create-an-impartial-picture">Changed And Added Lines Create an Impartial Picture</h2>
 
<h2 id="changed-and-added-lines-create-an-impartial-picture">Changed And Added Lines Create an Incomplete Picture</h2>
 
<p>In <a href="https://www.linuxfoundation.org/sites/main/files/publications/estimatinglinux.html"><em>Estimating the Total Cost of a Linux Distribution</em></a>, McPherson, Proffitt, and Hale-Evans write:</p>
 
<blockquote>
 
<p>Anyone who is familiar with kernel development, for instance, realizes that the highest man-power cost in its development is when code is deleted and modified. The amount of effort that goes into deleting and changing code, not just adding to it, is not reflected in the values associated with this estimate. Because in a collaborative development model, code is developed and then changed and deleted, the true value is far greater than the existing code base. Just think about the process: when a few lines of code are added to the kernel, for instance, many more have to be modified to be compatible with that change. The work that goes into understanding the dependencies and outcomes and then changing that code is not well represented in this study.</p>
 
</blockquote>
 
<p>Therefore, the process described herein, which ignores lines that are <em>deleted</em> (thus streamlining and improving code), also ignores a fundamental tenant of software development. Namely, making code smaller, more expressive, and more concise yields better-designed and more easily maintainable software. While the process herein <em>can</em> produce a clear list of code whose known introduction is directly attributable to Hellwig, the analysis produced by this process does not do justice to the full weight of the contributions made by Hellwig, since removed code is outright ignored in this process.</p>
 
<p>However, we can consider this process above to have found a bare minimum of Hellwig's contributions that appear in VMware's partial source release.</p>
 
<h1 id="further-analysis-of-additional-examples">Further Analysis of Additional Examples</h1>
 
<p>Separately from the analysis above, Hellwig identified a specific list of eight critical C functions to which he specifically recalls contributing, and near-equivalents were found in in VMware's ESXi 5.5 product.</p>
 
<p>In this additional analysis, we used CCFinderX in a different way <a href="#fn6" class="footnoteRef" id="fnref6"><sup>6</sup></a>. In these tests, I confine the code tests to specific small sections of code that were previously identified by human analysis as similar. In this way, I used CCFinderX to confirm with computational analysis what was already obvious to the human eye.</p>
 
<p>As expected, the ratio of similarity between the Hellwig's implementation and the corresponding implementation found in VMware's ESXi 5.5 product are quite high. As show in the table below, these particular functions show a incredibly high degree of similarity.</p>
 
<table>
 
<thead>
0 comments (0 inline, 0 general)