Deployed 6dec498 with MkDocs version: 1.5.3

SparkContext/index.html

@@ -11,8 +11,10 @@
 <span class=c1>// recursive = false</span>
 <span class=n>addFile</span><span class=p>(</span>
 <span class=w>  </span><span class=n>path</span><span class=p>:</span><span class=w> </span><span class=nc>String</span><span class=p>):</span><span class=w> </span><span class=nc>Unit</span>
-</code></pre></div> <p><code>addFile</code> creates a Hadoop <code>Path</code> from the given <code>path</code>. For a no-schema path, <code>addFile</code> converts it to a canonical form.</p> <p><code>addFile</code> prints out the following WARN message to the logs and exits.</p> <div class=highlight><pre><span></span><code>File with &#39;local&#39; scheme is not supported to add to file server, since it is already available on every node.
-</code></pre></div> <p><code>addFile</code>...FIXME</p> <p>In the end, <code>addFile</code> adds the file to the <a href=#addedFiles>addedFiles</a> internal registry (with the current timestamp):</p> <ul> <li> <p>For new files, <code>addFile</code> prints out the following INFO message to the logs, <a href=../Utils/#fetchFile>fetches the file</a> (to the root directory and without using the cache) and <a href=#postEnvironmentUpdate>postEnvironmentUpdate</a>.</p> <div class=highlight><pre><span></span><code>Added file [path] at [key] with timestamp [timestamp]
+</code></pre></div> <p>Firstly, <code>addFile</code> validate the schema of given <code>path</code>. For a no-schema path, <code>addFile</code> converts it to a canonical form. For a local schema path, <code>addFile</code> prints out the following WARN message to the logs and exits.</p> <p><div class=highlight><pre><span></span><code>File with &#39;local&#39; scheme is not supported to add to file server, since it is already available on every node.
+</code></pre></div> And for other schema path, <code>addFile</code> creates a Hadoop Path from the given path.</p> <p><code>addFile</code> Will validate the URL if the path is an HTTP, HTTPS or FTP URI.</p> <p><code>addFile</code> Will throw <code>SparkException</code> with below message if path is local directories but not in local mode.</p> <div class=highlight><pre><span></span><code>addFile does not support local directories when not running local mode.
+</code></pre></div> <p><code>addFile</code> Will throw <code>SparkException</code> with below message if path is directories but not turn on <code>recursive</code> flag.</p> <div class=highlight><pre><span></span><code>Added file $hadoopPath is a directory and recursive is not turned on.
+</code></pre></div> <p>In the end, <code>addFile</code> adds the file to the <a href=#addedFiles>addedFiles</a> internal registry (with the current timestamp):</p> <ul> <li> <p>For new files, <code>addFile</code> prints out the following INFO message to the logs, <a href=../Utils/#fetchFile>fetches the file</a> (to the root directory and without using the cache) and <a href=#postEnvironmentUpdate>postEnvironmentUpdate</a>.</p> <div class=highlight><pre><span></span><code>Added file [path] at [key] with timestamp [timestamp]
 </code></pre></div> </li> <li> <p>For files that were already added, <code>addFile</code> prints out the following WARN message to the logs:</p> <div class=highlight><pre><span></span><code>The path [path] has been added already. Overwriting of added paths is not supported in the current version.
 </code></pre></div> </li> </ul> <p><code>addFile</code> is used when:</p> <ul> <li><code>SparkContext</code> is <a href=../SparkContext-creating-instance-internals/#addFile>created</a></li> </ul> <h3 id=listfiles><span id=listFiles> listFiles<a class=headerlink href=#listfiles title="Permanent link">&para;</a></h3> <div class=highlight><pre><span></span><code><span class=n>listFiles</span><span class=p>():</span><span class=w> </span><span class=nc>Seq</span><span class=p>[</span><span class=nc>String</span><span class=p>]</span>
 </code></pre></div> <p><code>listFiles</code> is the <a href=#addedFiles>files added</a>.</p> <h3 id=addedfiles-internal-registry><span id=addedFiles> addedFiles Internal Registry<a class=headerlink href=#addedfiles-internal-registry title="Permanent link">&para;</a></h3> <div class=highlight><pre><span></span><code><span class=n>addedFiles</span><span class=p>:</span><span class=w> </span><span class=nc>Map</span><span class=p>[</span><span class=nc>String</span><span class=p>,</span><span class=w> </span><span class=nc>Long</span><span class=p>]</span>