<p>I have a file which is an XML representation of some data that is taken from a Web service and cached locally within a Web Application. The idea being is that this data is <em>very</em> static, but just <em>might</em> change. So I have set it up to cache to a file, and stuck a monitor against it to check if it has been deleted. Once deleted, the file will be refreshed from its source and rebuilt.</p>

<p>I am now running in to problems though, because obviously in a multi-threaded environment it falls over as it is trying to access the data when it is still reading/writing the file.</p>

<p>This is confusing me, because I added a object to lock against, and this is always locked during read/write. It was my understanding that attempted access from other threads would be told to "wait" until the lock was released?</p>

<p>Just to let you know, I am real new to multi-threaded development, so I am totally willing to accept this is a screw up on my part :)</p>

<ul>
<li><strong>Am I missing something?</strong></li>
<li><strong>What is the best file access strategy in a multi-threaded environment?</strong>

<hr></li>
</ul>

<h3>Edit</h3>

<p>Sorry - I should have said this is using <strong>ASP.NET 2.0</strong> :)</p>

[{'answer_id': 41312, 'author': 'Ubiguchi', 'author_id': 2562, 'author_profile': 'https://Stackoverflow.com/users/2562', 'pm_score': 1, 'selected': False, 'text': '<p>If you\'re locking on a object stored as a <em>static</em> then the lock should work for all threads in the same Application Domain, but perhaps you need to upload a code sample so we can have a look at the offending lines.</p>\n\n<p>That said, one thought would be to check if IIS is configured to run in <a href="http://www.microsoft.com/technet/prodtechnol/WindowsServer2003/Library/IIS/57fd3d02-9b05-4b3f-8c3a-f326ff895ab8.mspx?mfr=true" rel="nofollow noreferrer">Web Garden</a> mode (i.e. more than 1 process executing your application) which would break your locking logic. While you could fix such a situation with a mutex it\'d be easier to reconfigure your application to execute in a single process, although you\'d be wise to check the performance before and after messing with the web garden settings as it can potentially affect performance.</p>\n'}, {'answer_id': 41351, 'author': 'Community', 'author_id': -1, 'author_profile': 'https://Stackoverflow.com/users/-1', 'pm_score': 1, 'selected': False, 'text': '<p>You could maybe create the file with a temporary name ("data.xml_TMP"), and when it\'s ready change the name to what it is supposed to be. That way, no other process will be accessing it before it is ready.</p>\n'}, {'answer_id': 41559, 'author': 'Eric Z Beard', 'author_id': 1219, 'author_profile': 'https://Stackoverflow.com/users/1219', 'pm_score': 4, 'selected': True, 'text': '<p>Here is the code that I use to make sure a file is not locked by another process.  It\'s not 100% foolproof, but it gets the job done most of the time:</p>\n\n<pre><code>    /// &lt;summary&gt;\n    /// Blocks until the file is not locked any more.\n    /// &lt;/summary&gt;\n    /// &lt;param name="fullPath"&gt;&lt;/param&gt;\n    bool WaitForFile(string fullPath)\n    {\n        int numTries = 0;\n        while (true)\n        {\n            ++numTries;\n            try\n            {\n                // Attempt to open the file exclusively.\n                using (FileStream fs = new FileStream(fullPath,\n                    FileMode.Open, FileAccess.ReadWrite, \n                    FileShare.None, 100))\n                {\n                    fs.ReadByte();\n\n                    // If we got this far the file is ready\n                    break;\n                }\n            }\n            catch (Exception ex)\n            {\n                Log.LogWarning(\n                   "WaitForFile {0} failed to get an exclusive lock: {1}", \n                    fullPath, ex.ToString());\n\n                if (numTries &gt; 10)\n                {\n                    Log.LogWarning(\n                        "WaitForFile {0} giving up after 10 tries", \n                        fullPath);\n                    return false;\n                }\n\n                // Wait for the lock to be released\n                System.Threading.Thread.Sleep(500);\n            }\n        }\n\n        Log.LogTrace("WaitForFile {0} returning true after {1} tries",\n            fullPath, numTries);\n        return true;\n    }\n</code></pre>\n\n<p>Obviously you can tweak the timeouts and retries to suit your application.  I use this to process huge FTP files that take a while to be written.</p>\n'}, {'answer_id': 113659, 'author': 'Rob Cooper', 'author_id': 832, 'author_profile': 'https://Stackoverflow.com/users/832', 'pm_score': 1, 'selected': False, 'text': '<p>OK, I have been working on this and ended up creating a stress-test module to basically hammer the crap out of my code from several threads (<a href="https://stackoverflow.com/questions/41568/whats-the-best-way-to-unit-test-from-multiple-threads">See Related Question</a>).</p>\n\n<p>It was much easier from this point on to find holes in my code. It turns out that my code wasn\'t actually far off, but there was a certain logic path that it could enter in to which basically caused read/write operations to stack up, meaning if they didn\'t get cleared in time, it would go boom!</p>\n\n<p>Once I took that out, ran my stress test again, all worked fine!</p>\n\n<p><strong>So, I didn\'t really do anything <em>special</em> in my file access code, just ensured I used <a href="http://msdn.microsoft.com/en-us/library/c5kehkcz(VS.85).aspx" rel="nofollow noreferrer"><code>lock</code></a> statements where appropriate (i.e. when reading or writing).</strong></p>\n'}, {'answer_id': 32701350, 'author': 'gmail user', 'author_id': 344394, 'author_profile': 'https://Stackoverflow.com/users/344394', 'pm_score': 0, 'selected': False, 'text': '<p>How about using <code>AutoResetEvent</code> to communicate between threads? I created a console app which creates roughly 8 GB file in <code>createfile</code> method and then copy that file in <code>main</code> method</p>\n\n<pre><code> static AutoResetEvent waitHandle = new AutoResetEvent(false);\n    static string filePath=@"C:\\Temp\\test.txt";\n    static string fileCopyPath=@"C:\\Temp\\test-copy.txt";\n    static void Main(string[] args)\n    {\n        Console.WriteLine("in main method");\n        Console.WriteLine();\n        Thread thread = new Thread(createFile);\n        thread.Start();\n\n        Console.WriteLine("waiting for file to be processed ");\n        Console.WriteLine();\n        waitHandle.WaitOne();\n        Console.WriteLine();\n\n        File.Copy(filePath, fileCopyPath);\n        Console.WriteLine("file copied ");\n\n    }\n\n\n    static void createFile()\n    {\n\n        FileStream fs= File.Create(filePath);            \n        Console.WriteLine("start processing a file "+DateTime.Now);\n        Console.WriteLine();\n        using (StreamWriter sw = new StreamWriter(fs))\n        {\n            for (long i = 0; i &lt; 300000000; i++)\n            {\n                sw.WriteLine("The value of i is " + i);\n\n            }\n        }\n        Console.WriteLine("file processed " + DateTime.Now);\n        Console.WriteLine();\n\n        waitHandle.Set();\n    }\n</code></pre>\n'}]
['https://Stackoverflow.com/questions/41290', 'https://Stackoverflow.com', 'https://Stackoverflow.com/users/832/']