<p>What are the best practices around creating flat file database structures in PHP?</p>

<p>A lot of more matured PHP flat file frameworks out there which I attempt to implement SQL-like query syntax which is over the top for my purposes in most cases. (I would just use a database at that point).</p>

<p>Are there any elegant tricks out there to get good performance and features with a small code overhead?</p>

[{'answer_id': 87, 'author': 'saint_groceon', 'author_id': 59, 'author_profile': 'https://Stackoverflow.com/users/59', 'pm_score': 4, 'selected': False, 'text': '<p>One framework I\'m considering would be for a blogging platform. Since just about any possible view of data you would want would be sorted by date, I was thinking about this structure:</p>\n\n<p>One directory per content node:</p>\n\n<pre><code>./content/YYYYMMDDHHMMSS/\n</code></pre>\n\n<p>Subdirectories of each node including  </p>\n\n<pre><code>/tags  \n/authors  \n/comments  \n</code></pre>\n\n<p>As well as simple text files in the node directory for pre- and post-rendered content and the like.</p>\n\n<p>This would allow a simple PHP <a href="http://us3.php.net/glob" rel="noreferrer"><code>glob()</code></a> call (and probably a reversal of the result array) to query on just about anything within the content structure:</p>\n\n<pre><code>glob("content/*/tags/funny");  \n</code></pre>\n\n<p>Would return paths including all articles tagged "funny".</p>\n'}, {'answer_id': 154, 'author': 'w-ll', 'author_id': 146637, 'author_profile': 'https://Stackoverflow.com/users/146637', 'pm_score': 7, 'selected': True, 'text': "<p>Well, what is the nature of the flat databases.  Are they large or small.  Is it simple arrays with arrays in them?  if its something simple say userprofiles built as such:</p>\n<pre><code>$user = array(&quot;name&quot; =&gt; &quot;bob&quot;, \n              &quot;age&quot; =&gt; 20,\n              &quot;websites&quot; =&gt; array(&quot;example.com&quot;,&quot;bob.example.com&quot;,&quot;bob2.example.com&quot;),\n              &quot;and_one&quot; =&gt; &quot;more&quot;);\n</code></pre>\n<p>and to save or update the <em>db record</em> for that user.</p>\n<pre><code>$dir = &quot;../userdata/&quot;;  //make sure to put it bellow what the server can reach.\nfile_put_contents($dir.$user['name'],serialize($user));\n</code></pre>\n<p>and to load the <em>record</em> for the user</p>\n<pre><code>function &amp;get_user($name){\n    return unserialize(file_get_contents(&quot;../userdata/&quot;.$name));\n}\n</code></pre>\n<p>but again this implementation will vary on the application and nature of the database you need.</p>\n"}, {'answer_id': 6454, 'author': 'yukondude', 'author_id': 726, 'author_profile': 'https://Stackoverflow.com/users/726', 'pm_score': 6, 'selected': False, 'text': '<p>You might consider <a href="http://www.sqlite.org/" rel="noreferrer">SQLite</a>. It\'s almost as simple as flat files, but you do get a SQL engine for querying. It <a href="http://ca3.php.net/sqlite" rel="noreferrer">works well with PHP</a> too.</p>\n'}, {'answer_id': 90542, 'author': 'Jason', 'author_id': 7391, 'author_profile': 'https://Stackoverflow.com/users/7391', 'pm_score': 3, 'selected': False, 'text': '<p>If you\'re going to use a flat file to persist data, use XML to structure the data. PHP has a <a href="http://uk.php.net/xml" rel="noreferrer">built-in XML parser</a>.</p>\n'}, {'answer_id': 90800, 'author': 'ofaurax', 'author_id': 15209, 'author_profile': 'https://Stackoverflow.com/users/15209', 'pm_score': 3, 'selected': False, 'text': "<p>If you want a human-readable result, you can also use this type of file :</p>\n\n<pre><code>ofaurax|27|male|something|\nanother|24|unknown||\n...\n</code></pre>\n\n<p>This way, you have only one file, you can debug it (and manually fix) easily, you can add fields later (at the end of each line) and the PHP code is simple (for each line, split according to |).</p>\n\n<p>However, the drawbacks is that you should parse the entire file to search something (if you have millions of entry, it's not fine) and you should handle the separator in data (for example if the nick is WaR|ordz).</p>\n"}, {'answer_id': 111609, 'author': 'Mez', 'author_id': 20010, 'author_profile': 'https://Stackoverflow.com/users/20010', 'pm_score': 5, 'selected': False, 'text': "<p>In my opinion, using a &quot;Flat File Database&quot; in the sense you're meaning (and the answer you've accepted) isn't necessarily the best way to go about things. First of all, using <code>serialize()</code> and <code>unserialize()</code> can cause MAJOR headaches if someone gets in and edits the file (they can, in fact, put arbitrary code in your &quot;database&quot; to be run each time.)</p>\n<p>Personally, I'd say - why not look to the future? There have been so many times that I've had issues because I've been creating my own &quot;proprietary&quot; files, and the project has exploded to a point where it needs a database, and I'm thinking &quot;you know, I wish I'd written this for a database to start with&quot; - because the refactoring of the code takes way too much time and effort.</p>\n<p>From this I've learnt that future proofing my application so that when it gets bigger I don't have to go and spend days refactoring is the way to go forward. How do I do this?</p>\n<p>SQLite. It works as a database, uses SQL, and is pretty easy to change over to MySQL (especially if you're using abstracted classes for database manipulation like I do!)</p>\n<p>In fact, especially with the &quot;accepted answer&quot;'s method, it can drastically cut the memory usage of your app (you don't have to load all the &quot;RECORDS&quot; into PHP)</p>\n"}, {'answer_id': 242850, 'author': 'Ryan McCue', 'author_id': 2575, 'author_profile': 'https://Stackoverflow.com/users/2575', 'pm_score': 3, 'selected': False, 'text': '<p>Here\'s the code we use for Lilina:</p>\n\n<pre><code>&lt;?php\n/**\n * Handler for persistent data files\n *\n * @author Ryan McCue &lt;cubegames@gmail.com&gt;\n * @package Lilina\n * @version 1.0\n * @license http://opensource.org/licenses/gpl-license.php GNU Public License\n */\n\n/**\n * Handler for persistent data files\n *\n * @package Lilina\n */\nclass DataHandler {\n    /**\n     * Directory to store data.\n     *\n     * @since 1.0\n     *\n     * @var string\n     */\n    protected $directory;\n\n    /**\n     * Constructor, duh.\n     *\n     * @since 1.0\n     * @uses $directory Holds the data directory, which the constructor sets.\n     *\n     * @param string $directory \n     */\n    public function __construct($directory = null) {\n        if ($directory === null)\n            $directory = get_data_dir();\n\n        if (substr($directory, -1) != \'/\')\n            $directory .= \'/\';\n\n        $this-&gt;directory = (string) $directory;\n    }\n\n    /**\n     * Prepares filename and content for saving\n     *\n     * @since 1.0\n     * @uses $directory\n     * @uses put()\n     *\n     * @param string $filename Filename to save to\n     * @param string $content Content to save to cache\n     */\n    public function save($filename, $content) {\n        $file = $this-&gt;directory . $filename;\n\n        if(!$this-&gt;put($file, $content)) {\n            trigger_error(get_class($this) . " error: Couldn\'t write to $file", E_USER_WARNING);\n            return false;\n        }\n\n        return true;\n    }\n\n    /**\n     * Saves data to file\n     *\n     * @since 1.0\n     * @uses $directory\n     *\n     * @param string $file Filename to save to\n     * @param string $data Data to save into $file\n     */\n    protected function put($file, $data, $mode = false) {\n        if(file_exists($file) &amp;&amp; file_get_contents($file) === $data) {\n            touch($file);\n            return true;\n        }\n\n        if(!$fp = @fopen($file, \'wb\')) {\n            return false;\n        }\n\n        fwrite($fp, $data);\n        fclose($fp);\n\n        $this-&gt;chmod($file, $mode);\n        return true;\n\n    }\n\n    /**\n     * Change the file permissions\n     *\n     * @since 1.0\n     *\n     * @param string $file Absolute path to file\n     * @param integer $mode Octal mode\n     */\n    protected function chmod($file, $mode = false){\n        if(!$mode)\n            $mode = 0644;\n        return @chmod($file, $mode);\n    }\n\n    /**\n     * Returns the content of the cached file if it is still valid\n     *\n     * @since 1.0\n     * @uses $directory\n     * @uses check() Check if cache file is still valid\n     *\n     * @param string $id Unique ID for content type, used to distinguish between different caches\n     * @return null|string Content of the cached file if valid, otherwise null\n     */\n    public function load($filename) {\n        return $this-&gt;get($this-&gt;directory . $filename);\n    }\n\n    /**\n     * Returns the content of the file\n     *\n     * @since 1.0\n     * @uses $directory\n     * @uses check() Check if file is valid\n     *\n     * @param string $id Filename to load data from\n     * @return bool|string Content of the file if valid, otherwise null\n     */\n    protected function get($filename) {\n        if(!$this-&gt;check($filename))\n            return null;\n\n        return file_get_contents($filename);\n    }\n\n    /**\n     * Check a file for validity\n     *\n     * Basically just a fancy alias for file_exists(), made primarily to be\n     * overriden.\n     *\n     * @since 1.0\n     * @uses $directory\n     *\n     * @param string $id Unique ID for content type, used to distinguish between different caches\n     * @return bool False if the cache doesn\'t exist or is invalid, otherwise true\n     */\n    protected function check($filename){\n        return file_exists($filename);\n    }\n\n    /**\n     * Delete a file\n     *\n     * @param string $filename Unique ID\n     */\n    public function delete($filename) {\n        return unlink($this-&gt;directory . $filename);\n    }\n}\n\n?&gt;\n</code></pre>\n\n<p>It stores each entry as a separate file, which we found is efficient enough for use (no unneeded data is loaded and it\'s faster to save).</p>\n'}, {'answer_id': 13670880, 'author': 'siliconrockstar', 'author_id': 761967, 'author_profile': 'https://Stackoverflow.com/users/761967', 'pm_score': 3, 'selected': False, 'text': '<p>IMHO, you have two... er, three options if you want to avoid homebrewing something:</p>\n<ol>\n<li><strong>SQLite</strong></li>\n</ol>\n<p>If you\'re familiar with PDO, you can install a PDO driver that supports SQLite.  Never used it, but I have used PDO a ton with MySQL.  I\'m going to give this a shot on a current project.</p>\n<ol start="2">\n<li><strong>XML</strong></li>\n</ol>\n<p>Done this many times for relatively small amounts of data.  <a href="http://us1.php.net/manual/en/intro.xmlreader.php" rel="nofollow noreferrer">XMLReader</a> is a lightweight, read-forward, cursor-style class.  <a href="http://us1.php.net/manual/en/intro.simplexml.php" rel="nofollow noreferrer">SimpleXML</a> makes it simple to read an XML document into an object that you can access just like any other class instance.</p>\n<ol start="3">\n<li><strong>JSON</strong> (update)</li>\n</ol>\n<p>Good option for smallish amounts of data, just read/write file and json_decode/json_encode. Not sure if PHP offers a structure to navigate a JSON tree without loading it all in memory though.</p>\n'}, {'answer_id': 13960906, 'author': 'jpcrevoisier', 'author_id': 1916991, 'author_profile': 'https://Stackoverflow.com/users/1916991', 'pm_score': 3, 'selected': False, 'text': '<p>I have written two simple functions designed to store data in a file. You can judge for yourself if it\'s useful in this case.\nThe point is to save a php variable (if it\'s either an array a string or an object) to a file.</p>\n\n<pre><code>&lt;?php\nfunction varname(&amp;$var) {\n    $oldvalue=$var;\n    $var=\'AAAAB3NzaC1yc2EAAAABIwAAAQEAqytmUAQKMOj24lAjqKJC2Gyqhbhb+DmB9eDDb8+QcFI+QOySUpYDn884rgKB6EAtoFyOZVMA6HlNj0VxMKAGE+sLTJ40rLTcieGRCeHJ/TI37e66OrjxgB+7tngKdvoG5EF9hnoGc4eTMpVUDdpAK3ykqR1FIclgk0whV7cEn/6K4697zgwwb5R2yva/zuTX+xKRqcZvyaF3Ur0Q8T+gvrAX8ktmpE18MjnA5JuGuZFZGFzQbvzCVdN52nu8i003GEFmzp0Ny57pWClKkAy3Q5P5AR2BCUwk8V0iEX3iu7J+b9pv4LRZBQkDujaAtSiAaeG2cjfzL9xIgWPf+J05IQ==\';\n    foreach($GLOBALS as $var_name =&gt; $value) {\n        if ($value === \'AAAAB3NzaC1yc2EAAAABIwAAAQEAqytmUAQKMOj24lAjqKJC2Gyqhbhb+DmB9eDDb8+QcFI+QOySUpYDn884rgKB6EAtoFyOZVMA6HlNj0VxMKAGE+sLTJ40rLTcieGRCeHJ/TI37e66OrjxgB+7tngKdvoG5EF9hnoGc4eTMpVUDdpAK3ykqR1FIclgk0whV7cEn/6K4697zgwwb5R2yva/zuTX+xKRqcZvyaF3Ur0Q8T+gvrAX8ktmpE18MjnA5JuGuZFZGFzQbvzCVdN52nu8i003GEFmzp0Ny57pWClKkAy3Q5P5AR2BCUwk8V0iEX3iu7J+b9pv4LRZBQkDujaAtSiAaeG2cjfzL9xIgWPf+J05IQ==\')\n        {\n            $var=$oldvalue;\n            return $var_name;\n        }\n    }\n    $var=$oldvalue;\n    return false;\n}\n\nfunction putphp(&amp;$var, $file=false)\n    {\n    $varname=varname($var);\n    if(!$file)\n    {\n        $file=$varname.\'.php\';\n    }\n    $pathinfo=pathinfo($file);\n    if(file_exists($file))\n    {\n        if(is_dir($file))\n        {\n            $file=$pathinfo[\'dirname\'].\'/\'.$pathinfo[\'basename\'].\'/\'.$varname.\'.php\';\n        }\n    }\n    file_put_contents($file,\'&lt;?php\'."\\n\\$".$varname.\'=\'.var_export($var, true).";\\n");\n    return true;\n}\n</code></pre>\n'}, {'answer_id': 14149357, 'author': 'Michael Burt', 'author_id': 1947156, 'author_profile': 'https://Stackoverflow.com/users/1947156', 'pm_score': 3, 'selected': False, 'text': '<p>Just pointing out a potential problem with a flat file database with this type of system:</p>\n\n<pre><code>data|some text|more data\n\nrow 2 data|bla hbalh|more data\n</code></pre>\n\n<p>...etc</p>\n\n<p>The problem is that the cell data contains a "|" or a "\\n" then the data will be lost. Sometimes it would be easier to split by combinations of letters that most people wouldn\'t use.</p>\n\n<p>For example:</p>\n\n<p>Column splitter: <code>#$% (Shift+345)</code></p>\n\n<p>Row splitter: <code>^&amp;* (Shift+678)</code></p>\n\n<p>Text file: <code>test data#$%blah blah#$%^&amp;*new row#$%new row data 2</code></p>\n\n<p>Then use: <code>explode("#$%", $data); use foreach, the explode again to separate columns</code></p>\n\n<p>Or anything along these lines. Also, I might add that flat file databases are good for systems with small amounts of data (ie. less than 20 rows), but become huge memory hogs for larger databases.</p>\n'}, {'answer_id': 16339974, 'author': 'omran', 'author_id': 2343402, 'author_profile': 'https://Stackoverflow.com/users/2343402', 'pm_score': 3, 'selected': False, 'text': '<p>This one is inspiring as a practical solution:<br>\n<a href="https://github.com/mhgolkar/FlatFire" rel="noreferrer">https://github.com/mhgolkar/FlatFire</a><br>\nIt uses multiple strategies to handling data...<br>\n[Copied from Readme File]</p>\n\n<h3>Free or Structured or Mixed</h3>\n\n<pre><code>- STRUCTURED\nRegular (table, row, column) format.\n[DATABASE]\n/   \\\nTX  TableY\n    \\_____________________________\n    |ROW_0 Colum_0 Colum_1 Colum_2|\n    |ROW_1 Colum_0 Colum_1 Colum_2|\n    |_____________________________|\n- FREE\nMore creative data storing. You can store data in any structure you want for each (free) element, its similar to storing an array with a unique "Id".\n[DATABASE]\n/   \\\nEX  ElementY (ID)\n    \\________________\n    |Field_0 Value_0 |\n    |Field_1 Value_1 |\n    |Field_2 Value_2 |\n    |________________|\nrecall [ID]: get_free("ElementY") --&gt; array([Field_0]=&gt;Value_0,[Field_1]=&gt;Value_1...\n- MIXD (Mixed)\nMixed databases can store both free elements and tables.If you add a table to a free db or a free element to a structured db, flat fire will automatically convert FREE or SRCT to MIXD database.\n[DATABASE]\n/   \\\nEX  TY\n</code></pre>\n'}]
['https://Stackoverflow.com/questions/85', 'https://Stackoverflow.com', 'https://Stackoverflow.com/users/59/']