答:
Can other developers in the team read LINQ?

If not then don't use it or one of two things will happen:

Your code will be unmaintainable

You'll be stuck with maintaining all your code and everything which relies on it

A for each loop is perfect for iterating through a list but if that's not what you what to do then don't use one.

Title: for vs. foreach vs. LINQ

Body:
When I write code in Visual Studio, ReSharper (God bless it!) often suggests me to change my old-school for loop in the more compact foreach form.

And often, when I accept this change, ReSharper goes a step forward, and suggests me to change it again, in a shiny LINQ form.

So, I wonder: are there some real advantages, in these improvements? In pretty simple code execution, I cannot see any speed boost (obviously), but I can see the code becoming less and less readable... So I wonder: is it worth it?


问:
for vs. foreach vs. LINQ
When I write code in Visual Studio, ReSharper (God bless it!) often suggests me to change my old-school for loop in the more compact foreach form.

And often, when I accept this change, ReSharper goes a step forward, and suggests me to change it again, in a shiny LINQ form.

So, I wonder: are there some real advantages, in these improvements? In pretty simple code execution, I cannot see any speed boost (obviously), but I can see the code becoming less and less readable... So I wonder: is it worth it?

答:
for vs. foreach

There is a common confusion that those two constructs are very similar and that both are interchangeable like this:

```<code>foreach (var c in collection)
{
    DoSomething(c);
}
</code>```

and:

```<code>for (var i = 0; i < collection.Count; i++)
{
    DoSomething(collection[i]);
}
</code>```

The fact that both keywords start by the same three letters doesn't mean that semantically, they are similar. This confusion is extremely error-prone, especially for beginners. Iterating through a collection and doing something with the elements is done with foreach; for doesn't have to and shouldn't be used for this purpose, unless you really know what you're doing.

Let's see what's wrong with it with an example. At the end, you'll find the full code of a demo application used to gather the results.

In the example, we are loading some data from the database, more precisely the cities from Adventure Works, ordered by name, before encountering "Boston". The following SQL query is used:

```<code>select distinct [City] from [Person].[Address] order by [City]
</code>```

The data is loaded by ListCities() method which returns an IEnumerable<string>. Here is what foreach looks like:

```<code>foreach (var city in Program.ListCities())
{
    Console.Write(city + " ");

    if (city == "Boston")
    {
        break;
    }
}
</code>```

Let's rewrite it with a for, assuming that both are interchangeable:

```<code>var cities = Program.ListCities();
for (var i = 0; i < cities.Count(); i++)
{
    var city = cities.ElementAt(i);

    Console.Write(city + " ");

    if (city == "Boston")
    {
        break;
    }
}
</code>```

Both return the same cities, but there is a huge difference.

When using foreach, ListCities() is called one time and yields 47 items.

When using for, ListCities() is called 94 times and yields 28153 items overall.

What happened?

IEnumerable is lazy. It means that it will do the work only at the moment when the result is needed. Lazy evaluation is a very useful concept, but has some caveats, including the fact that it's easy to miss the moment(s) where the result will be needed, especially in the cases where the result is used multiple times.

In a case of a foreach, the result is requested only once. In a case of a for as implemented in the incorrectly written code above, the result is requested 94 times, i.e. 47 × 2:

Every time cities.Count() is called (47 times),

Every time cities.ElementAt(i) is called (47 times).

Querying a database 94 times instead of one is terrible, but not the worse thing which may happen. Imagine, for example, what would happen if the select query would be preceded by a query which also inserts a row in the table. Right, we would have for which will call the database 2,147,483,647 times, unless it hopefully crashes before.

Of course, my code is biased. I deliberately used the laziness of IEnumerable and wrote it in a way to repeatedly call ListCities(). One can note that a beginner will never do that, because:

The IEnumerable<T> doesn't have the property Count, but only the method Count(). Calling a method is scary, and one can expect its result to not be cached, and not suitable in a for (; ...; ) block.

The indexing is unavailable for IEnumerable<T> and it's not obvious to find the ElementAt LINQ extension method.

Probably most beginners would just convert the result of ListCities() to something they are familiar with, like a List<T>.

```<code>var cities = Program.ListCities();
var flushedCities = cities.ToList();
for (var i = 0; i < flushedCities.Count; i++)
{
    var city = flushedCities[i];

    Console.Write(city + " ");

    if (city == "Boston")
    {
        break;
    }
}
</code>```

Still, this code is very different from the foreach alternative. Again, it gives the same results, and this time the ListCities() method is called only once, but yields 575 items, while with foreach, it yielded only 47 items.

The difference comes from the fact that ToList() causes all data to be loaded from the database. While foreach requested only the cities before "Boston", the new for requires all cities to be retrieved and stored in memory. With 575 short strings, it probably doesn't make much difference, but what if we were retrieving only few rows from a table containing billions of records?

So what is foreach, really?

foreach is closer to a while loop. The code I previously used:

```<code>foreach (var city in Program.ListCities())
{
    Console.Write(city + " ");

    if (city == "Boston")
    {
        break;
    }
}
</code>```

can be simply replaced by:

```<code>using (var enumerator = Program.ListCities().GetEnumerator())
{
    while (enumerator.MoveNext())
    {
        var city = enumerator.Current;
        Console.Write(city + " ");

        if (city == "Boston")
        {
            break;
        }
    }
}
</code>```

Both produce the same IL. Both have the same result. Both have the same side effects. Of course, this while can be rewritten in a similar infinite for, but it would be even longer and error-prone. You're free to choose the one you find more readable.

Want to test it yourself? Here's the full code:

```<code>using System;
using System.Collections.Generic;
using System.Data;
using System.Data.SqlClient;
using System.Diagnostics;
using System.Linq;

public class Program
{
    private static int countCalls;

    private static int countYieldReturns;

    public static void Main()
    {
        Program.DisplayStatistics("for", Program.UseFor);
        Program.DisplayStatistics("for with list", Program.UseForWithList);
        Program.DisplayStatistics("while", Program.UseWhile);
        Program.DisplayStatistics("foreach", Program.UseForEach);

        Console.WriteLine("Press any key to continue...");
        Console.ReadKey(true);
    }

    private static void DisplayStatistics(string name, Action action)
    {
        Console.WriteLine("--- " + name + " ---");

        Program.countCalls = 0;
        Program.countYieldReturns = 0;

        var measureTime = Stopwatch.StartNew();
        action();
        measureTime.Stop();

        Console.WriteLine();
        Console.WriteLine();
        Console.WriteLine("The data was called {0} time(s) and yielded {1} item(s) in {2} ms.", Program.countCalls, Program.countYieldReturns, measureTime.ElapsedMilliseconds);
        Console.WriteLine();
    }

    private static void UseFor()
    {
        var cities = Program.ListCities();
        for (var i = 0; i < cities.Count(); i++)
        {
            var city = cities.ElementAt(i);

            Console.Write(city + " ");

            if (city == "Boston")
            {
                break;
            }
        }
    }

    private static void UseForWithList()
    {
        var cities = Program.ListCities();
        var flushedCities = cities.ToList();
        for (var i = 0; i < flushedCities.Count; i++)
        {
            var city = flushedCities[i];

            Console.Write(city + " ");

            if (city == "Boston")
            {
                break;
            }
        }
    }

    private static void UseForEach()
    {
        foreach (var city in Program.ListCities())
        {
            Console.Write(city + " ");

            if (city == "Boston")
            {
                break;
            }
        }
    }

    private static void UseWhile()
    {
        using (var enumerator = Program.ListCities().GetEnumerator())
        {
            while (enumerator.MoveNext())
            {
                var city = enumerator.Current;
                Console.Write(city + " ");

                if (city == "Boston")
                {
                    break;
                }
            }
        }
    }

    private static IEnumerable<string> ListCities()
    {
        Program.countCalls++;
        using (var connection = new SqlConnection("Data Source=mframe;Initial Catalog=AdventureWorks;Integrated Security=True"))
        {
            connection.Open();

            using (var command = new SqlCommand("select distinct [City] from [Person].[Address] order by [City]", connection))
            {
                using (var reader = command.ExecuteReader(CommandBehavior.SingleResult))
                {
                    while (reader.Read())
                    {
                        Program.countYieldReturns++;
                        yield return reader["City"].ToString();
                    }
                }
            }
        }
    }
}
</code>```

And the results:

--- for ---

  Abingdon Albany Alexandria Alhambra [...] Bonn Bordeaux Boston

The data was called 94 time(s) and yielded 28153 item(s).

--- for with list ---

  Abingdon Albany Alexandria Alhambra [...] Bonn Bordeaux Boston

The data was called 1 time(s) and yielded 575 item(s).

--- while ---

  Abingdon Albany Alexandria Alhambra [...] Bonn Bordeaux Boston

The data was called 1 time(s) and yielded 47 item(s).

--- foreach ---

  Abingdon Albany Alexandria Alhambra [...] Bonn Bordeaux Boston

The data was called 1 time(s) and yielded 47 item(s).

LINQ vs. traditional way

As for LINQ, you may want to learn functional programming (FP) - not C# FP stuff, but real FP language like Haskell. Functional languages have a specific way to express and present the code. In some situations, it is superior to non-functional paradigms.

FP is known being much superior when it comes to manipulating lists (list as a generic term, unrelated to List<T>). Given this fact, the ability to express C# code in a more functional way when it comes to lists is rather a good thing.

If you're not convinced, compare the readability of code written in both functional and non-functional ways in my previous answer on the subject.

ContentLicense: CC BY-SA 3.0

Title: for vs. foreach vs. LINQ

Body:
for vs. foreach

There is a common confusion that those two constructs are very similar and that both are interchangeable like this:

foreach (var c in collection)
{
    DoSomething(c);
}

and:

for (var i = 0; i < collection.Count; i++)
{
    DoSomething(collection[i]);
}

The fact that both keywords start by the same three letters doesn't mean that semantically, they are similar. This confusion is extremely error-prone, especially for beginners. Iterating through a collection and doing something with the elements is done with foreach; for doesn't have to and shouldn't be used for this purpose, unless you really know what you're doing.

Let's see what's wrong with it with an example. At the end, you'll find the full code of a demo application used to gather the results.

In the example, we are loading some data from the database, more precisely the cities from Adventure Works, ordered by name, before encountering "Boston". The following SQL query is used:

select distinct [City] from [Person].[Address] order by [City]

The data is loaded by ListCities() method which returns an IEnumerable<string>. Here is what foreach looks like:

foreach (var city in Program.ListCities())
{
    Console.Write(city + " ");

    if (city == "Boston")
    {
        break;
    }
}

Let's rewrite it with a for, assuming that both are interchangeable:

var cities = Program.ListCities();
for (var i = 0; i < cities.Count(); i++)
{
    var city = cities.ElementAt(i);

    Console.Write(city + " ");

    if (city == "Boston")
    {
        break;
    }
}

Both return the same cities, but there is a huge difference.

When using foreach, ListCities() is called one time and yields 47 items.

When using for, ListCities() is called 94 times and yields 28153 items overall.

What happened?

IEnumerable is lazy. It means that it will do the work only at the moment when the result is needed. Lazy evaluation is a very useful concept, but has some caveats, including the fact that it's easy to miss the moment(s) where the result will be needed, especially in the cases where the result is used multiple times.

In a case of a foreach, the result is requested only once. In a case of a for as implemented in the incorrectly written code above, the result is requested 94 times, i.e. 47 × 2:

Every time cities.Count() is called (47 times),

Every time cities.ElementAt(i) is called (47 times).

Querying a database 94 times instead of one is terrible, but not the worse thing which may happen. Imagine, for example, what would happen if the select query would be preceded by a query which also inserts a row in the table. Right, we would have for which will call the database 2,147,483,647 times, unless it hopefully crashes before.

Of course, my code is biased. I deliberately used the laziness of IEnumerable and wrote it in a way to repeatedly call ListCities(). One can note that a beginner will never do that, because:

The IEnumerable<T> doesn't have the property Count, but only the method Count(). Calling a method is scary, and one can expect its result to not be cached, and not suitable in a for (; ...; ) block.

The indexing is unavailable for IEnumerable<T> and it's not obvious to find the ElementAt LINQ extension method.

Probably most beginners would just convert the result of ListCities() to something they are familiar with, like a List<T>.

var cities = Program.ListCities();
var flushedCities = cities.ToList();
for (var i = 0; i < flushedCities.Count; i++)
{
    var city = flushedCities[i];

    Console.Write(city + " ");

    if (city == "Boston")
    {
        break;
    }
}

Still, this code is very different from the foreach alternative. Again, it gives the same results, and this time the ListCities() method is called only once, but yields 575 items, while with foreach, it yielded only 47 items.

The difference comes from the fact that ToList() causes all data to be loaded from the database. While foreach requested only the cities before "Boston", the new for requires all cities to be retrieved and stored in memory. With 575 short strings, it probably doesn't make much difference, but what if we were retrieving only few rows from a table containing billions of records?

So what is foreach, really?

foreach is closer to a while loop. The code I previously used:

foreach (var city in Program.ListCities())
{
    Console.Write(city + " ");

    if (city == "Boston")
    {
        break;
    }
}

can be simply replaced by:

using (var enumerator = Program.ListCities().GetEnumerator())
{
    while (enumerator.MoveNext())
    {
        var city = enumerator.Current;
        Console.Write(city + " ");

        if (city == "Boston")
        {
            break;
        }
    }
}

Both produce the same IL. Both have the same result. Both have the same side effects. Of course, this while can be rewritten in a similar infinite for, but it would be even longer and error-prone. You're free to choose the one you find more readable.

Want to test it yourself? Here's the full code:

using System;
using System.Collections.Generic;
using System.Data;
using System.Data.SqlClient;
using System.Diagnostics;
using System.Linq;

public class Program
{
    private static int countCalls;

    private static int countYieldReturns;

    public static void Main()
    {
        Program.DisplayStatistics("for", Program.UseFor);
        Program.DisplayStatistics("for with list", Program.UseForWithList);
        Program.DisplayStatistics("while", Program.UseWhile);
        Program.DisplayStatistics("foreach", Program.UseForEach);

        Console.WriteLine("Press any key to continue...");
        Console.ReadKey(true);
    }

    private static void DisplayStatistics(string name, Action action)
    {
        Console.WriteLine("--- " + name + " ---");

        Program.countCalls = 0;
        Program.countYieldReturns = 0;

        var measureTime = Stopwatch.StartNew();
        action();
        measureTime.Stop();

        Console.WriteLine();
        Console.WriteLine();
        Console.WriteLine("The data was called {0} time(s) and yielded {1} item(s) in {2} ms.", Program.countCalls, Program.countYieldReturns, measureTime.ElapsedMilliseconds);
        Console.WriteLine();
    }

    private static void UseFor()
    {
        var cities = Program.ListCities();
        for (var i = 0; i < cities.Count(); i++)
        {
            var city = cities.ElementAt(i);

            Console.Write(city + " ");

            if (city == "Boston")
            {
                break;
            }
        }
    }

    private static void UseForWithList()
    {
        var cities = Program.ListCities();
        var flushedCities = cities.ToList();
        for (var i = 0; i < flushedCities.Count; i++)
        {
            var city = flushedCities[i];

            Console.Write(city + " ");

            if (city == "Boston")
            {
                break;
            }
        }
    }

    private static void UseForEach()
    {
        foreach (var city in Program.ListCities())
        {
            Console.Write(city + " ");

            if (city == "Boston")
            {
                break;
            }
        }
    }

    private static void UseWhile()
    {
        using (var enumerator = Program.ListCities().GetEnumerator())
        {
            while (enumerator.MoveNext())
            {
                var city = enumerator.Current;
                Console.Write(city + " ");

                if (city == "Boston")
                {
                    break;
                }
            }
        }
    }

    private static IEnumerable<string> ListCities()
    {
        Program.countCalls++;
        using (var connection = new SqlConnection("Data Source=mframe;Initial Catalog=AdventureWorks;Integrated Security=True"))
        {
            connection.Open();

            using (var command = new SqlCommand("select distinct [City] from [Person].[Address] order by [City]", connection))
            {
                using (var reader = command.ExecuteReader(CommandBehavior.SingleResult))
                {
                    while (reader.Read())
                    {
                        Program.countYieldReturns++;
                        yield return reader["City"].ToString();
                    }
                }
            }
        }
    }
}

And the results:

--- for ---

  Abingdon Albany Alexandria Alhambra [...] Bonn Bordeaux Boston

The data was called 94 time(s) and yielded 28153 item(s).

--- for with list ---

  Abingdon Albany Alexandria Alhambra [...] Bonn Bordeaux Boston

The data was called 1 time(s) and yielded 575 item(s).

--- while ---

  Abingdon Albany Alexandria Alhambra [...] Bonn Bordeaux Boston

The data was called 1 time(s) and yielded 47 item(s).

--- foreach ---

  Abingdon Albany Alexandria Alhambra [...] Bonn Bordeaux Boston

The data was called 1 time(s) and yielded 47 item(s).

LINQ vs. traditional way

As for LINQ, you may want to learn functional programming (FP) - not C# FP stuff, but real FP language like Haskell. Functional languages have a specific way to express and present the code. In some situations, it is superior to non-functional paradigms.

FP is known being much superior when it comes to manipulating lists (list as a generic term, unrelated to List<T>). Given this fact, the ability to express C# code in a more functional way when it comes to lists is rather a good thing.

If you're not convinced, compare the readability of code written in both functional and non-functional ways in my previous answer on the subject.

ContentLicense: CC BY-SA 4.0


问:
What is a "task pool"?
I have searched for many websites for a definition of "task pool", but most descriptions are very vague. Anywhere I can find a precise definition of it?

More details.

Are dependencies allowed in task pool?

Can a running task exchange data with other running tasks?

To phrase conversely, are tasks permitted to accept data only prior to start and permitted to publish results only after the task has finished?

What keeps track of the temporary resources (memory, CPU etc) that are used by tasks?

How to define weights that will encourage efficient task assignment?

For example, if one task needs to pass a lot of data to the next task, it may make sense to run the next task immediately following the first task, and as close to the first thread / CPU / machine / cluster (locality) as possible

What about tasks that are not known (do not exist) in the beginning, but were "created" later?

答:
A very generic answer is that a task pool is very similar to a priority queue.

Depending on the implementation:

You may have an infinite number of subscribers

Dependencies can be resolved and sorted according to priority (for instance, a low priority task that depends on the same thing that a high priority task does might get processed before a medium priority task)

Tasks can be fed from a static memory pool, or allocate their own, depending

Tasks can be preempted if a higher priority task is added and can typically yield on a voluntary basis as well.

A much simpler version of a task pool can be realized by taking 'priority' out of 'priority queue', where you more or less have a FIFO scheduler that keeps track of time and resources.

I think the term is a little too colloquial to define, clearly, outside the context of the platform, language and even task at hand - no pun intended.

ContentLicense: CC BY-SA 2.5

Title: What is a "task pool"?


问:
I want to build a Virtual Machine, are there any good references?
I'm looking to build a Virtual Machine as a platform independent way to run some game code (essentially scripting).

The Virtual Machines that I'm aware of in games are rather old: Infocom's Z-Machine, LucasArts' SCUMM, id Software's Quake 3. As a .net Developer, I'm familiar with the CLR and looked into the CIL Instructions to get an overview of what you actually implement on a VM Level (vs. the language level). I've also dabbled a bit in 6502 Assembler during the last year.

The thing is, now that I want¹ to implement one, I need to dig a bit deeper. I know that there are stack based and register based VMs, but I don't really know which one is better at what and if there are more or hybrid approaches. I need to deal with memory management, decide which low level types are part of the VM and need to understand why stuff like ldstr works the way it does.

My only reference book (apart from the Z-Machine stuff) is the CLI Annotated Standard, but I wonder if there is a better, more general/fundamental lecture for VMs? Basically something like the Dragon Book, but for VMs? I'm aware of Donald Knuth's Art of Computer Programming which uses a register-based VM, but I'm not sure how applicable that series still is, especially since it's still unfinished?

Clarification: The goal is to build a specialized VM. For example, Infocom's Z-Machine contains OpCodes for setting the Background Color or playing a sound. So I need to figure out how much goes into the VM as OpCodes vs. the compiler that takes a script (language TBD) and generates the bytecode from it, but for that I need to understand what I'm really doing.

¹ I know, modern technology would allow me to just interpret a high level scripting language on the fly. But where is the fun in that? :) It's also a bit hard to google because Virtual Machines is nowadays often associated with VMWare-type OS Virtualization...

答:
I'd start by checking Lua.  Both as a sample implementation, and as a very usable VM/language out of the box if you finally decide not to roll your own.

The source code is very readble, and there's also the Annotated source code.  And some Design documents written by the main author, Roberto Ierusalimschy.

Finally, if you choose to use it instead of your own, you'll find that it's been long a favorite among game developers, and there's a very high performance JIT implementation.

About stack- vs register-based, I think stack-based VMs are easier to design, but the compiler can be more complex.  As the Iesualimschy paper notes, Lua was one of the first register-based language VMs, but afterwards several others have sprouted, Most notably, LLVM, Dalvik, and some modern JavaScript VMs.


问:
How to experience gradual improvement of knowledge while a newbie does .NET maintenance programming?
I started my career as a software developer about 6 months ago. This is my first job, and I am the only developer in this company. I gained .NET knowledge by self study and also by doing some university projects.  Our systems have old foundations based on an earlier version of .NET, and I'm starting to feel that I am not improving since I am a maintenance programmer here. Everything is old and my manager is not really taking any chances on gradually improving the software.

What is your opinion? What should I do? I am newbie and also work hard to find my way through. There is no other developer, not even a senior one to help me here.
I need your advice on my situation. And one last thing, can I get a new job with doing maintenance programming? I mean don't managers say that you do not have the experience of developing a new software from scratch? I feel redundant, what do I do?

答:
Well, my answer is twofold.

No matter where you work or who you work with, you are ultimately responsible for your own career development.  There are many ways to improve your coding - here are some tips.  If you are on your own, especially at the beginning of your career, that makes it difficult.  I would look hard for a mentor or mentors.  That means attending relevant user-groups in your area, taking a course (online or in person) or maybe paying a CS student, or professor, or older programmer to tutor you.

For many, if not most people, bug fixing is an important precursor to fixing large problems and eventually performing program design.  Even if you don't do maintenance programming, it's important to have done it, because most organizations spend most of their "development" dollars doing maintenance.  Someone suggested "Clean Code" and "Refactoring" which are great suggestions, but it's important to notice that writing maintainable code has it's own (small) genre of literature - it's important!

Bug fixing gives you a tour of other people's worst decisions - which is important!  But make sure to also take the time to look at the parts of the code that work, not just the broken parts.  The areas that you don't have to touch are probably well written and will provide you with some positive examples.

I agree with others not to stay in pure-maintenance forever (unless you want to).  Writing new code is a lot of fun, and generally a much more positive experience.  But maintenance programming certainly won't hurt you, and having a few years of it under your belt will definitely make you a better programmer.


问:
Is it conceivable to have millions of lists of data in memory in Python?
I have over the last 30 days been developing a Python application that utilizes a MySQL database of information (specifically about Norwegian addresses) to perform address validation and correction. The database contains approximately 2.1 million rows (43 columns) of data and occupies 640MB of disk space.

I'm thinking about speed optimizations, and I've got to assume that when validating 10,000+ addresses, each validation running up to 20 queries to the database, networking is a speed bottleneck.

I haven't done any measuring or timing yet, and I'm sure there are simpler ways of speed optimizing the application at the moment, but I just want to get the experts' opinions on how realistic it is to load this amount of data into a row-of-rows structure in Python. Also, would it even be any faster? Surely MySQL is optimized for looking up records among vast amounts of data, so how much help would it even be to remove the networking step? Can you imagine any other viable methods of removing the networking step?

The location of the MySQL server will vary, as the application might well be run from a laptop at home or at the office, where the server would be local.

答:
Doc Brown's answer and Mason Wheeler's comment are probably the best, most practical ways to go here.  That said, it's an interesting conceptual question, so I'm inclined to comment.

Also, would it even be any faster?  Surely MySQL is optimized for looking up records among vast amounts of data, so how much help would it even be to remove the networking step?

We're essentially talking about two things: having the data client-side, and having "custom code" to do the lookups in your data set.  The advantages I see for this arrangement:

Decreased round trip time; with performance, you'd have to measure to be sure of anything, but if it turns out that your scenario is mostly dominated by waiting for 200 000 queries to be passed over the network (and back), having the data on the client would definitely help.  You'd also be offloading the work to the client, so this might be useful in a scenario where you're expecting many clients (say thousands or more), at the same time.

Deep knowledge about the data and fine-tuning your data structures; MySQL's engine is certainly very optimised, but it doesn't really know what data it is storing (it doesn't really care if it's addresses or phone numbers or patient information).  You can take advantage of this fact, and come up with clever ways to encode the information for quick lookup.  E.g. if we know that most Norwegian streets start at some house number, and then they are mostly contiguous up to the last number in that street, you can create a pretty efficient lookup mechanism.  Is the house number between the min and max for that street, and not on the exception list?  Then it's a valid address for that street.  Coincidentally, this type of thing could be done on the DB as well, the main difference would be the ability to choose and tune the data structures yourself (e.g. having a custom hash algorithm, etc).

The second point could probably be used to come up with a smaller representation of your DB.  I'm not sure how normalised your data is at the moment, but generally address DBs tend to have much of the address in text, which is then mostly duplicated many times over and over.  This is not a very efficient way of storing things, and if this is true of your DB, it could probably be represented in a much more compact file (a quick test might be to ZIP or RAR the DB to see if there's much duplication in it, although the distribution of data might mean that these algorithms don't help as much as they could).  Aside - if street numbers are being mixed with street names as text, I'm guessing a lot could be done to optimise the DB for indexing.

TL;DR: Yes, theoretically, you could get some benefits out of moving the data to the client side.  Practically, however, you're probably far better off just batching your queries, or using any of the other great ideas already posted.


问:
How do I properly credit authors of software released under the MIT license?
I have modified the source code of an MIT licensed project, and I have added new classes to it as well. Please correct me if I am wrong, but I think that it is legal to add my copyright notice above the license and remove the other one.  But how should I attribute the contribution of the previous authors? Should I use a separate file? There are also some HTML files with no license or copyright notice in them which I also modified. Do I have to handle them differently?

My question is different than this question in that I have also modified some of the files of the project I am extending.

Update

Although the suggestion to remove the copyright notice sounds weird, what I had in mind when I first posted this, is that the authors should not be responsible if I add something malicious to their code. This should not be a problem since MIT license includes a disclaimer.

答:
I am not a lawyer, but according to the MIT license:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

Therefore you may not remove the existing copyright notice. You may only add your copyright notice and license terms to portions of code that you own the copyright to, which are your new or modified code.


问:
NoSQL as file meta database
I am trying to implement a virtual file system structure in front of an object storage (Openstack). For availability reasons we initially chose Cassandra, however while designing file system data model, it looked like a tree structure similar to a relational model. Here is the dilemma for availability and partition tolerance we need NoSQL, but our data model is relational.

The intended file system must be able to handle filtered search based on date, name etc. as fast as possible. So what path should i take? Stick to relational with some indexing mechanism backed by 3 rd tools like Apache Solr or dig deeper into NoSQL and find a suitable model and database satisfying the model?

P.S: Currently from NoSQL Cassandra or MongoDB are choices proposed by my colleagues.

答:
As I could see from your questions is, that you first should think about what you need and what is NoSql made for.

NoSQL - made for huge amount of distributed data. Scales well from performance an amount of data. and depending in the Type of NoSQL system you can just put and get objects very fast, or also do long running jobs on the distributed data, it's not made for fast search/queries.

Search -  it's just made for searching references to data fast. Depending on system it scales well with data and performance. its not made for query huge amount of data nor data relations.

RDBMS - it's made to store data which have relations to each other. System scales depending on system itself and based on your data design. means even the fastest rdbms solutions could perform out with wrong data/query design. It's not made for fast search on huge amount of data. And scaling is depending of rdmbs product, it's not a feature by default.

So if you want to search data from a huge amount, choose the system that is made for it: Search Engine.

If you want to store huge amount of data, where data needs to be distributed (because of amount) and performance of getting data should be independent from amount of data: choose a NoSQL System.

If you don't have that much data,but data needs to have relations to each other, then choose a RDBMS, and think well of you data design.

If you need a Search which stored the index distributed, combine a search product with distributed storage (filesystem).

If you need a distributed filesystem, have a look at Apache Hadoop.

If you need a NoSQL System like Google Big Table, which is somehow compareable to a RDBMS, have a look at Apache Hbase:  hbase.apache.org or Hypertable.

Cassandra is more like Amazon Dynamo. It's distributed, but more far away from rdbms.

and MongoDB, is more close to LotusNotes. It's a good storage for Documents/Objects.

And MAIN POINT to solve the problem, think solution dependent, means: do NOT think like RDBMS when using a NoSQL System, you need to think in that specific NoSQL System (they are all very different from each other).

Title: NoSQL as file meta database


问:
Strategy for avoiding duplicate object ids for data shared across devices using iCloud
I have a data intensive iOS app that is not using CoreData nor does it support iCloud synching (yet). All of my objects are created with unique keys. I use a simple long long initialized with the current time. Then as I need a new key I increment the value by 1. This has all worked well for a few years with the app running isolated on a single device.

Now I want to add support for automatic data sync across devices using iCloud. As my app is written, there is the possibility that two objects created on two different devices could end up with the same key. I need to avoid this possibility.

I'm looking for ideas for solving this issue. I have a few requirements that the solution must meet:

1) The key needs to remain a single integral data type. Converting all existing keys to a compound key or to a string or other type would affect the entire code base and likely result in more bugs than it's worth.

2) The solution can't depend on an Internet connection. A user must be able to run the app and add data even with no Internet connection. The data should still resolve properly later when the data syncs through iCloud once a connection is available. I'll accept one exception to this rule. If no other option is available, I may be open to requiring an Internet connection the first time the app's data is initialized.

One idea I have been toying around with in my head is logically splitting the integer key into two parts. The high 4 or 5 bits could be used as some sort of device id while the rest represents the actual key. The fuzzy part is figuring out how to come up with non-conflicting device ids that fit in a few bits. This should be viable since I don't need to deal will millions of devices. I just need to deal with the few devices that would be shared by a given iCloud account.

I'm open to suggestions. Thanks.

Update:

After giving this some more thought, I've decided to take the short term hit and do this the right way. Using a GUID is definitely the best long term option for this. Using this approach eliminates the need to do any of the various options to deal with handing out key ranges or translating keys once an Internet connection is made.

My original requirement prevented the GUID option because neither SQLite nor iOS support a simple, 128-bit integer type. Since I have a large existing code base that is written such that my keys are simple integer types, it will require a major refactoring of both the code and the database schema.

In the end, I've decided that the short term pain of doing this refactoring out weighs the long term issues of dealing with a klunky solution. Taking the hit now gives me a much simpler and less error prone solution that will last the long term.

答: Your other option would be to continue assigning keys on the client as you currently do but then when you sync, you re-generate the key on the server which makes it unique. You will have to remember to update the key on any child entities to make sure they link back up correctly.</p>


问:
How is IntelliJ better than Eclipse?
I know there have been questions like What is your favorite editor/IDE?, but none of them have answered this question: Why spend the money  on IntelliJ when Eclipse is free?

I'm personally a big IntelliJ fan, but I haven't really tried Eclipse. I've used IntelliJ for projects that were Java, JSP, HTML/CSS, Javascript, PHP, and Actionscript, and the latest version, 9, has been excellent for all of them.

Many coworkers in the past have told me that they believe Eclipse to be "pretty much the same" as IntelliJ, but, to counter that point, I've occasionally sat behind a developer using Eclipse who's seemed  comparably inefficient (to accomplish roughly the same task), and I haven't experienced this with IntelliJ. They may be on par feature-by-feature but features can be ruined by a poor user experience, and I wonder if it's possible that IntelliJ is easier to pick up and discover time-saving features.

For users who are already familiar with Eclipse, on top of the real cost of IntelliJ, there is also the cost of time spent learning the new app. Eclipse gets a lot of users who simply don't want to spend $250 on an IDE.

If IntelliJ really could help my team be more productive, how could I sell it to them? For those users who've tried both, I'd be very interested in specific pros or cons either way.

答:
I don't know of solid feature-by-feature blows, but one thing that I've discovered with in-and-out use of the IntelliJ environment for about a decade is that the design teams at JetBrains have done a great job of both having the IDE "index the world" as well as keep the indexing and autocomplete functions both responsive as well as consistent - I don't typically think about it, but with the responsiveness plus the number of keyboard shortcuts (and the ability to map more), IntelliJ makes it very easy to build "muscle memory" when using the editor.

It's not a small thing when you can write code, autocomplete, compile and test without your fingers leaving the keyboard, and in a way that doesn't feel like a stereotypical sluggish Java app.

Title: How is IntelliJ better than Eclipse?


答:
The difference between LINQ and foreach really boils down to two different programming styles: imperative and declarative.

Imperative: in this style you tell the computer "do this... now do this... now do this now do this". You feed it a program one step at a time.

Declarative: in this style you tell the computer what you want the result to be and let it figure out how to get there.

A classic example of these two styles is comparing assembly code (or C) with SQL. In assembly you give instructions (literally) one at a time. In SQL you express how to join data together and what result you want from that data.

A nice side-effect of declarative programming is that it tends to be a bit higher level. This allows for the platform to evolve underneath you without you having to change your code. For instance:

```<code>var foo = bar.Distinct();
</code>```

What is happening here? Is Distinct using one core? Two? Fifty? We don't know and we don't care. The .NET devs could rewrite it at any time, as long as it continues to perform the same purpose our code could just magically get faster after a code update.

This is the power of functional programming. And the reason that you'll find that code in languages like Clojure, F# and C# (written with a functional programming mindset) is often 3x-10x smaller than it's imperative counterparts.

Finally I like the declarative style because in C# most of the time this allows me to write code that does not mutate the data. In the above example, Distinct() does not change bar, it returns a new copy of the data. This means that whatever bar is, and where ever it came from, it not suddenly changed.

So like the other posters are saying, learn functional programming. It'll change your life. And if you can, do it in a true functional programming language. I prefer Clojure, but F# and Haskell are also excellent choices.

Title: for vs. foreach vs. LINQ

Body:
The difference between LINQ and foreach really boils down to two different programming styles: imperative and declarative.

Imperative: in this style you tell the computer "do this... now do this... now do this now do this". You feed it a program one step at a time.

Declarative: in this style you tell the computer what you want the result to be and let it figure out how to get there.

A classic example of these two styles is comparing assembly code (or C) with SQL. In assembly you give instructions (literally) one at a time. In SQL you express how to join data together and what result you want from that data.

A nice side-effect of declarative programming is that it tends to be a bit higher level. This allows for the platform to evolve underneath you without you having to change your code. For instance:

var foo = bar.Distinct();

What is happening here? Is Distinct using one core? Two? Fifty? We don't know and we don't care. The .NET devs could rewrite it at any time, as long as it continues to perform the same purpose our code could just magically get faster after a code update.

This is the power of functional programming. And the reason that you'll find that code in languages like Clojure, F# and C# (written with a functional programming mindset) is often 3x-10x smaller than it's imperative counterparts.

Finally I like the declarative style because in C# most of the time this allows me to write code that does not mutate the data. In the above example, Distinct() does not change bar, it returns a new copy of the data. This means that whatever bar is, and where ever it came from, it not suddenly changed.

So like the other posters are saying, learn functional programming. It'll change your life. And if you can, do it in a true functional programming language. I prefer Clojure, but F# and Haskell are also excellent choices.


问:
How do I correctly sub-license a library that is under the MIT license?
How do I correctly sub-license a library that is under the MIT license.

I am using and extending the library. The MIT license states that I am free to sub-license the library.

Can I simply state:

```<code><Software library> is copyright <original author> and licensed under the MIT license.

<orignal license>

Extensions to <Software library> are copyright <me> and licensed under the GPL license,
or commercial license if applicable.

<GPL, or commercial license>
</code>```

答:
So the first portion of your question with "how do I correctly sub-license" is answered in this former SO link: Can I re-license someone's MIT code.  Hat tip to Gallaecio for this link, and please note that SO users with 10k+ reputation can still see the link.

And the most relevant part of the answer there is:

What you can do is have more than one license on the software.

  So, you need to include the license file, but I'd add something about it not applying to the whole product. To be nice, you might want to point out where somebody can get a copy of the MIT-licensed starting point.

You need to include the GPL as well as the original license, and you need to make it clear that each individual file as a whole is covered by the GPL, as well as MIT for portions of it.

This is an additional SO question regarding MIT vs GPL licenses that I think is relevant to your question.

It's worth noting that when you sub-license that MIT licensed project into a GPL'd project that you are presenting the entire package as a GPL'd project, which includes the viral aspect of GPL.  The original project you're borrowing will remain available under the MIT license through whatever source you picked it up from.  The difference is that the package is now GPL'd as part of your project.

Arguably, if someone uses the MIT licensed project from your GPL'd project, they would need to consider it as GPL'd.  However, they could also just go back to the original source of the project under the MIT license and use it from there under an MIT license.  It's splitting hairs, and I think it's impossible to predict how it would play out in court.

The GPL FAQ has a good section on what it means for two licenses to be compatible.

This references the Free Software Foundation's comments on various licenses.  Their comments on the MIT license are: This is a lax, permissive non-copyleft free software license, compatible with the GNU GPL.  Which is a good thing from your question's point of view.

Here's a link to the GPL v3 license text

Here's a link to the MIT license text

FSF calls the MIT license the Expat license


问:
Obtain reference to Parent object during instantiation
I have a situation where a custom class is a property of another class.

What I need to be able to do, if it is possible at all, is obtain a reverse to the "parent" class (ie the the class that holds the current class as a property).

For Instance:

```<code>Public Class Class1
  ...
  public readonly property Prop11 as Class2
  public property Prop12 as String
  ...
End Class

Public Class Class2
  ...
  private _par as Class1
  private _var21 as string
  ...
  Public Sub New(...)
    me._par = ????
    ...
  End Sub
  public readonly property Prop21 as string
    Get
      return me._par.Prop12 & me._var21
    End Get
  End Property
  ...
End Class
</code>```

Ultimately, I am trying to access other properties within Class1 from Class2 as they do have substance for information from within Class2.  There are several other classes within Class1 that provide descriptive information to other classes contained within it as properties but the information is not extensible to all of the classes through Inheritance, as Class1 is being used as a resource bin for the property classes and the application itself.

Diagram, lazy design ;):

```<code>Application <- Class1.Prop12
Application <- Class1.Prop11.Prop21
</code>```

Question:

Is it possible to get a recursion through this design setup?

答:
I think your design is not optimal, that something is amiss. And the names you are using do not help either.

It's kind of a forced analogy but... Does a clock need to know who owns it to do its job? Does a pen need to know who owns it to its job? It's the same thing here. You can say that the clock does need some info about the owner to do its job, specifically the time for the alarm.

But this is not the case here. In your code, Class2 needs something from Class1 in order to even exist. One of its properties can't leave on it owns without Class1. If it was a method, I'd understand. But a property?

Anyway, alternatives:

If you do need to know the owner, pass it as constructor parameter.

Pass what the other classes need as parameter.

If it is a lot of information, then your Class1 is probably a composition of smaller objects. Break it and pass the little pieces as parameters to other class.

Use a 3rd class as a Helper. It'll use info from the other two classes to do what it's needed.

PS: If you could edit and make the name of classes/properties more meaningful, we can help you better.


答:
Please correct me if I am wrong, but I think that it is legal to add my copyright notice above the license and remove the other one.

You cannot delete the original copyright notice. That's out-and-out illegal.

Whether you can add your own notice is a bit dicey, and I am not a lawyer. What you have created falls into the category of "Derivative Works."

From US Copyright Office Circular 14: Derivative Works:

To be copyrightable, a derivative work must be different enough from the original to be regarded as a "new work" or must contain a substantial amount of new material. Making minor changes or additions of little substance to a preexisting work will not qualify the work as a new version for copyright purposes. The new material must be original and copyrightable in itself. Titles, short phrases, and format, for example, are not copyrightable.

If your changes do constitute copyrightable material, your copyright only pertains to your contributions. It does not pertain to the preexisting material.

Title: How do I properly credit authors of software released under the MIT license?

Body:
I have modified the source code of an MIT licensed project, and I have added new classes to it as well. Please correct me if I am wrong, but I think that it is legal to add my copyright notice above the license and remove the other one.  But how should I attribute the contribution of the previous authors? Should I use a separate file? There are also some HTML files with no license or copyright notice in them which I also modified. Do I have to handle them differently?

My question is different than this question in that I have also modified some of the files of the project I am extending.

Update

Although the suggestion to remove the copyright notice sounds weird, what I had in mind when I first posted this, is that the authors should not be responsible if I add something malicious to their code. This should not be a problem since MIT license includes a disclaimer.


答:
I don't have any specific resources to link you to at the moment, but I've researched a similar topic in the past and found the Smalltalk VM to be a good learning aid as well.  There are many academic papers and articles written about the byte codes used by Smalltalk, as well as writing interpreters and VMs to use that bytecode.  A Google search for smalltalk vm implementation or smalltalk bytecode interpreter should yield lots of reading material.

If you'd like to see some source code or try out an implementation I recommend either the Squeak or Pharo versions.

The related language/VM Self might also interest you, as Self is basically Smalltalk with prototype-based objects (similar to JavaScript).

Title: I want to build a Virtual Machine, are there any good references?

Body:
I'm looking to build a Virtual Machine as a platform independent way to run some game code (essentially scripting).

The Virtual Machines that I'm aware of in games are rather old: Infocom's Z-Machine, LucasArts' SCUMM, id Software's Quake 3. As a .net Developer, I'm familiar with the CLR and looked into the CIL Instructions to get an overview of what you actually implement on a VM Level (vs. the language level). I've also dabbled a bit in 6502 Assembler during the last year.

The thing is, now that I want¹ to implement one, I need to dig a bit deeper. I know that there are stack based and register based VMs, but I don't really know which one is better at what and if there are more or hybrid approaches. I need to deal with memory management, decide which low level types are part of the VM and need to understand why stuff like ldstr works the way it does.

My only reference book (apart from the Z-Machine stuff) is the CLI Annotated Standard, but I wonder if there is a better, more general/fundamental lecture for VMs? Basically something like the Dragon Book, but for VMs? I'm aware of Donald Knuth's Art of Computer Programming which uses a register-based VM, but I'm not sure how applicable that series still is, especially since it's still unfinished?

Clarification: The goal is to build a specialized VM. For example, Infocom's Z-Machine contains OpCodes for setting the Background Color or playing a sound. So I need to figure out how much goes into the VM as OpCodes vs. the compiler that takes a script (language TBD) and generates the bytecode from it, but for that I need to understand what I'm really doing.

¹ I know, modern technology would allow me to just interpret a high level scripting language on the fly. But where is the fun in that? :) It's also a bit hard to google because Virtual Machines is nowadays often associated with VMWare-type OS Virtualization...


答:
Well, your design has problems, that's certain, and you've obfuscated the problem a bit too much here.

What I can suggest, based on what you've given, is that, instead of this:

```<code>class1.Prop12.Prop21
</code>```

do this:

```<code>class1.WrappedProp
</code>```

where Class1 implements WrappedProp like this:

```<code>Public Class Class1
    ...
    Public readonly property WrappedProp as string
        Get
            return me.Prop12 & myChild.Var21
        End Get
    End Property
    ...
End Class
</code>```

That is to say that the "application" (or whatever object holds a reference to your class1/parent) should not access the child class (or, know about it at all for that matter). From what I can tell, what you're really doing is providing some kind of container or wrapper with your parent class, but not completely wrapping or containing the child type.

Title: Obtain reference to Parent object during instantiation

Body:
I have a situation where a custom class is a property of another class.

What I need to be able to do, if it is possible at all, is obtain a reverse to the "parent" class (ie the the class that holds the current class as a property).

For Instance:

```<code>Public Class Class1
  ...
  public readonly property Prop11 as Class2
  public property Prop12 as String
  ...
End Class

Public Class Class2
  ...
  private _par as Class1
  private _var21 as string
  ...
  Public Sub New(...)
    me._par = ????
    ...
  End Sub
  public readonly property Prop21 as string
    Get
      return me._par.Prop12 & me._var21
    End Get
  End Property
  ...
End Class
</code>```

Ultimately, I am trying to access other properties within Class1 from Class2 as they do have substance for information from within Class2.  There are several other classes within Class1 that provide descriptive information to other classes contained within it as properties but the information is not extensible to all of the classes through Inheritance, as Class1 is being used as a resource bin for the property classes and the application itself.

Diagram, lazy design ;):

```<code>Application <- Class1.Prop12
Application <- Class1.Prop11.Prop21
</code>```

Question:

Is it possible to get a recursion through this design setup?

Body:
Well, your design has problems, that's certain, and you've obfuscated the problem a bit too much here.

What I can suggest, based on what you've given, is that, instead of this:

class1.Prop12.Prop21

do this:

class1.WrappedProp

where Class1 implements WrappedProp like this:

Public Class Class1
    ...
    Public readonly property WrappedProp as string
        Get
            return me.Prop12 & myChild.Var21
        End Get
    End Property
    ...
End Class

That is to say that the "application" (or whatever object holds a reference to your class1/parent) should not access the child class (or, know about it at all for that matter). From what I can tell, what you're really doing is providing some kind of container or wrapper with your parent class, but not completely wrapping or containing the child type.


答:
I would start from analyzing of how [script] source code gets into your machine or runtime environment.

If you have something like in HTML documents onclick="dosomething();"> then you will need very fast compiler, bytecode execution speed does not really matter that much in this case.
If your use cases are closer to Java/.NET where you can afford full blown compilation then VM architecture and bytecode structure will be closer Java bytecodes or IL.

Another criteria is what I name as "glueness". Originally scripts were developed as glue languages - scripts just define the way of how to connect various native functions together (Perl, Python, Ruby, JS). In that case effectiveness of VM and bytecode is far less critical than in case of Java/.NET when most of your code are functions written in the language itself.

And the last major criteria I would use is extensibility of your language.
If you have plans to add to you language runtime many native objects/functions implemented in, say, C++ then your VM architecture should be "convenient" for integration with C++.
For example: if you plan to expose to script  C++ objects as they are then the only option for you will be reference counting as a heap management (like Python, see boost::python as an example of integration). If you plan to use moving/compacting heap/GC then it will be different story. Lua's way of adding native stuff into runtime is a bit tricky [for C++ developers].

In other words, try to define first your typical use case and it will be easier to suggest what to read for you.

Body:
I would start from analyzing of how [script] source code gets into your machine or runtime environment.

If you have something like in HTML documents <a onclick="dosomething();"> then you will need very fast compiler, bytecode execution speed does not really matter that much in this case.
If your use cases are closer to Java/.NET where you can afford full blown compilation then VM architecture and bytecode structure will be closer Java bytecodes or IL.

Another criteria is what I name as "glueness". Originally scripts were developed as glue languages - scripts just define the way of how to connect various native functions together (Perl, Python, Ruby, JS). In that case effectiveness of VM and bytecode is far less critical than in case of Java/.NET when most of your code are functions written in the language itself.

And the last major criteria I would use is extensibility of your language.
If you have plans to add to you language runtime many native objects/functions implemented in, say, C++ then your VM architecture should be "convenient" for integration with C++.
For example: if you plan to expose to script  C++ objects as they are then the only option for you will be reference counting as a heap management (like Python, see boost::python as an example of integration). If you plan to use moving/compacting heap/GC then it will be different story. Lua's way of adding native stuff into runtime is a bit tricky [for C++ developers].

In other words, try to define first your typical use case and it will be easier to suggest what to read for you.


答:
Well...I would suggest you consider re-thinking your design, but sure you can have two objects that each reference each other; that is trivial.

The real question is how do you want to handle the instantiation / initialization. Does one of the objects, the "parent" as you've opted to think about it, contain the "child", or does a third object get involved to coordinate the aggregation of "parent" and "child".

However, before you rush into this, a similar concept you might want to check out is the Visitor pattern, which is used for bigger things than this but in a granular sense hinges around one object being passed to another which then passes itself back to that object.

There is also Kent Beck's "self delegation" pattern; if instead of using properties to couple the two objects together you restructured your logic into one or more methods, you could have the "parent" pass itself into the "child"'s method(s) as a parameter as needed, thus avoiding back-coupling from the "child" to the "parent".

Going a little further out, depending on what this "parent" and "child" set up is meant to model, you might just be looking at a Composite implementation.


问:
How to deal with huge changes to a data specification?
We have a C program which uses a .h header file to describe the custom-made data structure we use. The data structure must follow the design in the functional specification. There is some diagram to describe the member of the data structures and it would be described a ANS.1 abstract syntax, and then the asn file would be compiled to a .h header file.

In order to enhance our system, a request to change the functional specification was approved and a new specification was made. ( We jump from 2.10 to 2.12 as we had skipped the upgrade of 2.11 ) Comparing 2.10 to 2.12, a lot of new stuff were added and some old stuff were removed. We need to use a new .asn to compile a new .h header file in order to follow the specification.

The problem is that, something useful were removed. For example some member of a data structure were removed and we have been using the member for logic in the program. The program can't even compiled with the new header file. The program have been up and running and we would like to avoid any major down time due to this upgrade.

P.S. The specification is not only for our company, it is a standard for the file structure of the files sending between companeis within the whole industry. Say, if one declare to use 2.12 spec, and other party will treat the file with 2.12 standard and discard any files which doesn't follow the standard.

P.S. Of course there is no option. The decision is made. However I am facing a problem that the c program was written by some other people and have complex logic. The program would be hugely damaged by the change of the header file. And I may have no idea some to restore peace and functionality to it. Any suggestion of how to check with each difference is appreciated.

答:
Versioning is your first line of defense.

If the passed data objects don't already have a version number embedded within them, you will want to add one.  Having a version ID will allow you to determine if it's the older spec with information you're interested in or the newer one.

Wrapping the data object | struct becomes your next line of defense.

Instead of just tweaking the existing code that reads the data object and pushes that into your application, create a thin layer of code that processes the data object and converts that to a structure you can pass around within your application.  At a broader level, you are decoupling your application from the specific version of the data object you receive.

Within this thin layer, you can add some logic that determines which version of the data object that you received.  If it's the later version, then you can add logic to replace the missing values or insert dummy values instead.

Wrapping the data object allows you to preserve the contract that you had between the layers of your code.  Likewise, you'll have flexibility in what versions of the data object you can now send out.

An example application stack might look like this:

```
Rest of the application code
    internal application data struct
data object processing layer
    received data objects (v2.10, v2.12)
```

Body:
Versioning is your first line of defense.

If the passed data objects don't already have a version number embedded within them, you will want to add one.  Having a version ID will allow you to determine if it's the older spec with information you're interested in or the newer one.

Wrapping the data object | struct becomes your next line of defense.

Instead of just tweaking the existing code that reads the data object and pushes that into your application, create a thin layer of code that processes the data object and converts that to a structure you can pass around within your application.  At a broader level, you are decoupling your application from the specific version of the data object you receive.

Within this thin layer, you can add some logic that determines which version of the data object that you received.  If it's the later version, then you can add logic to replace the missing values or insert dummy values instead.

Wrapping the data object allows you to preserve the contract that you had between the layers of your code.  Likewise, you'll have flexibility in what versions of the data object you can now send out.

An example application stack might look like this:

Rest of the application code
    internal application data struct
data object processing layer
    received data objects (v2.10, v2.12)


问:
Why Open-Source Code?
What reasons to companies have for open-sourcing libraries and applications?

Doing this may allow a developer to better understand the code, but could doing this allow people to find and exploit vulnerabilities in the library or application?

答:
There are myriad reasons why a company would put code out into the world for anyone to see and use:

Developer Relations - Putting out quality, useful code endears developers to the company.

Trial by Fire - If you have code out in the wild that is used by many people, you are likely to get loads of feedback about it. Bugs, areas to improve, and maybe vulnerabilities. It can lead to more robust products in the end as you find many things that a few developers may not have found on their own.

Free Developer Time - Plenty of developers are perfectly willing to spend time working on a project. By allowing developers to contribute, you get the benefit of free labor as well.

Growth - Putting out an API allows other developers to create an ecosystem around the product/platform/etc. which in turn can attract more people and grow the ecosystem.

Title: Why Open-Source Code?


答:
A good reason would be this:

If some 3rd party developer gets an awesome idea and integrates it in that code, it can easily boost the popularity of that company or that product; this is always desired, even if it means some (not a lot) additional security risks. It's a trade-off worth making.

Also, exposing some API of your code to literally everyone can be beneficial. The best example I can think of is what Google did with Google Maps. They can be integrated in virtually any web-page out there. This helped Google by making that service much more popular and attractive. I know this doesn't fit the criteria of publishing the code; but making an API completely public comes pretty close.

Title: Why Open-Source Code?

Body:
What reasons to companies have for open-sourcing libraries and applications?

Doing this may allow a developer to better understand the code, but could doing this allow people to find and exploit vulnerabilities in the library or application?


答:
While there are some great expositions already on the differences between for and foreach. There are some gross misrepresentations of the role of LINQ.

LINQ syntax is not just syntactic sugar giving a functional programming approximation to C#. LINQ provides Functional constructs including all the benefits thereof to C#. Combined with returning IEnumerable instead of IList, LINQ provides deferred execution of the iteration. What people typically do now is construct and return an IList from their functions like so

```<code>public IList<Foo> GetListOfFoo()
{
   var retVal=new List<Foo>();
   foreach(var foo in _myPrivateFooList)
   {
      if(foo.DistinguishingValue == check)
      {
         retVal.Add(foo);
      }
   }
   return retVal;
}
</code>```

Instead use the yield return syntax to create a deferred enumeration.

```<code>public IEnumerable<Foo> GetEnumerationOfFoo()
{
   //no need to create an extra list
   //var retVal=new List<Foo>();
   foreach(var foo in _myPrivateFooList)
   {
      if(foo.DistinguishingValue == check)
      {
         //yield the match compiler handles the complexity
         yield return foo;
      }
   }
   //no need for returning a list
   //return retVal;
}
</code>```

Now the enumeration won't occur until you ToList or iterate over it. And it only occurs as needed (here's an enumeration of Fibbonaci that doesn't have a stack overflow problem)

```<code>/**
Returns an IEnumerable of fibonacci sequence
**/
public IEnumerable<int> Fibonacci()
{
  int first, second = 1;
  yield return first;
  yield return second;
  //the 46th fibonacci number is the largest that
  //can be represented in 32 bits.
  for (int i = 3; i < 47; i++)
  {
    int retVal = first+second;
    first=second;
    second=retVal;
    yield return retVal;
  }
}
</code>```

Performing a foreach over the Fibonacci function will return the sequence of 46. If you want the 30th that's all that will be calculated

```<code>var thirtiethFib=Fibonacci().Skip(29).Take(1);
</code>```

Where we get to have a lot of fun is the support in the language for lambda expressions (combined with the IQueryable and IQueryProvider constructs, this allows for functional composition of queries against a variety of data sets, the IQueryProvider is responsible for interpreting the passed in expressions and creating and executing a query using the source's native constructs). I won't go into the nitty gritty details here, but there is a series of blog posts showing how to create a SQL Query Provider here

In summary, you should prefer returning IEnumerable over IList when the consumers of your function will perform a simple iteration. And use the capabilities of LINQ to defer execution of complex queries until they are needed.

Title: for vs. foreach vs. LINQ

Body:
While there are some great expositions already on the differences between for and foreach. There are some gross misrepresentations of the role of LINQ.

LINQ syntax is not just syntactic sugar giving a functional programming approximation to C#. LINQ provides Functional constructs including all the benefits thereof to C#. Combined with returning IEnumerable instead of IList, LINQ provides deferred execution of the iteration. What people typically do now is construct and return an IList from their functions like so

public IList<Foo> GetListOfFoo()
{
   var retVal=new List<Foo>();
   foreach(var foo in _myPrivateFooList)
   {
      if(foo.DistinguishingValue == check)
      {
         retVal.Add(foo);
      }
   }
   return retVal;
}

Instead use the yield return syntax to create a deferred enumeration.

public IEnumerable<Foo> GetEnumerationOfFoo()
{
   //no need to create an extra list
   //var retVal=new List<Foo>();
   foreach(var foo in _myPrivateFooList)
   {
      if(foo.DistinguishingValue == check)
      {
         //yield the match compiler handles the complexity
         yield return foo;
      }
   }
   //no need for returning a list
   //return retVal;
}

Now the enumeration won't occur until you ToList or iterate over it. And it only occurs as needed (here's an enumeration of Fibbonaci that doesn't have a stack overflow problem)

/**
Returns an IEnumerable of fibonacci sequence
**/
public IEnumerable<int> Fibonacci()
{
  int first, second = 1;
  yield return first;
  yield return second;
  //the 46th fibonacci number is the largest that
  //can be represented in 32 bits.
  for (int i = 3; i < 47; i++)
  {
    int retVal = first+second;
    first=second;
    second=retVal;
    yield return retVal;
  }
}

Performing a foreach over the Fibonacci function will return the sequence of 46. If you want the 30th that's all that will be calculated

var thirtiethFib=Fibonacci().Skip(29).Take(1);

Where we get to have a lot of fun is the support in the language for lambda expressions (combined with the IQueryable and IQueryProvider constructs, this allows for functional composition of queries against a variety of data sets, the IQueryProvider is responsible for interpreting the passed in expressions and creating and executing a query using the source's native constructs). I won't go into the nitty gritty details here, but there is a series of blog posts showing how to create a SQL Query Provider here

In summary, you should prefer returning IEnumerable over IList when the consumers of your function will perform a simple iteration. And use the capabilities of LINQ to defer execution of complex queries until they are needed.


问:
How to read reduce/shift conflicts in LR(1) DFA?
I am reading an explanation (awesome "Parsing Techniques" by D.Grune and C.J.H.Jacobs; p.293 in the 2nd edition) and I moved forward from my last question: How to get lookahead symbol when constructing LR(1) NFA for parser?

Now I have such "problem" (maybe not a problem, but rather need of confirmation from some more knowledgeable people).

The authors present state in LR(0) which has reduce/shift conflict. Then they build DFA for LR(1) for the same grammar. And now they say it does not have a conflict (lookaheads at the end):

```<code>S -> E .        eof
E -> E . - T    eof
E -> E . - T    -
</code>```

and there is an edge from this state labeled - but no labeled eof. Authors says, that on eof there will be reduce, on - there will be shift.

However eof is for shift as well (as lookahead). So my personal understanding of LR(1) DFA is this -- you can drop lookaheads for shifts, because they serve no purpose now -- shifts rely on input, not on lookaheads -- and after that, remove duplicates.

```<code>S -> E .        eof
E -> E . - T
</code>```

So the lookahead for reduce serves as input really, because at this stage (all required input is read) it is really incoming symbol right now. For shifts, the input symbols are on the edges.

So my question is this -- am I actually right about dropping lookaheads for shifts (after fully constructing DFA)?

答: Lookaheads in final form in DFA do not matter -- next symbol for shift is on the edge.</p>


问:
Should I use a parser generator or should I roll my own custom lexer and parser code?
What specific advantages and disadvantages of each way to working on a programming language grammar?

Why/When should I roll my own? Why/When should I use a generator?

答:
That depends entirely on what you need to parse. Can you roll your own faster than you could hit the learning curve of a lexer? Is the stuff to be parsed static enough that you won't regret the decision later? Do you find existing implementations overly complex? If so, have fun rolling your own, but only if you aren't ducking a learning curve.

Lately, I've come to really like the lemon parser, which is arguably the simplest and easiest that I've ever used. For the sake of making things easy to maintain, I just use that for most needs. SQLite uses it as well as some other notable projects.

But, I'm not at all interested in lexers, beyond them not getting in my way when I need to use one (hence, lemon). You might be, and if so, why not make one? I have a feeling you'll come back to using one that exists, but scratch the itch if you must :)


问:
Is "watermarking" code with random trailing whitespace a good way to detect plagiarism?
Consider this:

```<code>int f(int x)
{
    return 2 * x * x;
}
</code>```

and this

```<code>int squareAndDouble(int y)
{
    return 2*y*y;
}
</code>```

If you found these in independent bodies of code, you might give the two programmers the benefit of the doubt and assume they came up with more-or-less the same function independently. But look at the whitespace at the end of each line of code. Same pattern in both. Surely evidence of copying. On a larger piece of code, correlation of random whitespace at line ends would be irrefutable evidence of a shared origin.

Now aside from the obvious weaknesses: e.g. visible or obvious in some editors, easily removed, I was wondering if it was worth deploying something like this in my open source project. My industry has a history of companies ripping off open source projects.

答:
First off, if someone rips off an open source project, presumably to create a closed source project, you are presumably not going to have access to the source of the closed source project without a lawsuit and enough evidence to get a court order to be able to examine the source code.  That would tend to be a relatively high bar to clear before you can even get access to the source to compare the whitespace.

Assuming that you have the motivation to file suit and that you can gather enough evidence to suggest copying without using the source, though, it's not obvious to me how beneficial white space analysis would be.  If this is a real open source project that is successful enough to steal, it's likely that the white space will change pretty regularly over time as people submit new versions of code so the "watermark" will change over time.  If you're assuming that the company that is ripping off the code is going to the effort of changing variable names and the spacing of the meaningful source (which would make applying changes from the open source project very challenging), it seems likely that the company would quickly just add a step to remove trailing whitespace from their code.

Most companies (and developers) that steal code do so much more blatantly.  They're just going to copy and paste the working code into their project and hit compile.  They're not going to bother changing variable names or changing the name of a procedure or anything else that requires them to understand and test the changes.  There will generally be much, much bigger smoking guns than the number of spaces at the end of the line.


答: Since "Trim Whitespace on save" is a feature of many IDEs, I don't think that this is good for anything.</p>

Title: Is "watermarking" code with random trailing whitespace a good way to detect plagiarism?

Body:
Consider this:

```<code>int f(int x)
{
    return 2 * x * x;
}
</code>```

and this

```<code>int squareAndDouble(int y)
{
    return 2*y*y;
}
</code>```

If you found these in independent bodies of code, you might give the two programmers the benefit of the doubt and assume they came up with more-or-less the same function independently. But look at the whitespace at the end of each line of code. Same pattern in both. Surely evidence of copying. On a larger piece of code, correlation of random whitespace at line ends would be irrefutable evidence of a shared origin.

Now aside from the obvious weaknesses: e.g. visible or obvious in some editors, easily removed, I was wondering if it was worth deploying something like this in my open source project. My industry has a history of companies ripping off open source projects.


答:
No.

Whitespace is insufficient to demonstrate a watermark as you're suggesting.

Running the code through a style formatter that changes tabs to spaces or vice versa and part of your watermark has been changed.  Additionally, based upon the style rules that are in place, trailing space could be truncated in order to comply with the style rule.

Presuming you could prove the "theft", what would you do?  You've made the code open-source, so you're giving away the source code.  And how would you gain access to their source code in the first place?


答:
It could also be an indication of using the same crummy editor that doesn't clean up line endings.

I found a few thousand lines of un-commented assembly code in a project I was working on over a decade ago.  I wondered at the time if that meant it was plagiarized code with the comments stripped out.

Ultimately, this is a legal question and the best answer is probably, "it depends on the jurisdiction."  Copyright laws between the US and UK are similar, I don't know about other countries, and certainly there are additional difficulties if the copyright his held in one country and infringed in another.

Using patterns of spacing at the end of lines is an interesting idea though.  Some editors (like IntelliJ and Emacs) will automatically truncate all trailing whitespace for you, so I don't know how effective it would be.

Here are Wikipedia's guidelines (not legal advice, just what works for them).


答:
I don't believe it'll be of any good, if the platform/language uses an formatting system (like those formatters found in the Delphi/FreePascal ecosystem, the pretty-print feature of VB.NET, the C# formatter on VS, etc)...

So if the code had passed by those tools, and considering that the tool is on its default settings, all watermarking is gone.

Some editors even have features to remove whitespaces at the end of a line (I think that the expression is "trailing blanks" - english is not my first language).

If the platform/language developers does not make much use of IDEs with autoformatting features, the watermarking can - in theory - work.

But I would not put much faith on just that.


答: Companies release code as open-source when that code isn't their core business. Facebook doesn't sell software, it sells a global social network. You can take 100% of Facebook's code, run it on your own servers, and still not be any sort of competition to them. So what do they have to lose? The reason they've beaten out rival social networks isn't because they have a super-efficient patented memory caching algorithm or the best PHP templating engine. The code isn't the only thing here.</p>

Title: Why Open-Source Code?


问:
Necessary Infrastructure for large project with many components communicating through IPCs
I have a fairly in depth question which probably doesn't have an exact answer.

As a software engineer, I am usually tasked with working on a program or project with minimal understanding of how other components or programs in the project interact with each other.  When one program fails in a sea of multiple components and processes, what infrastructure elements are necessary to ensure that the problem can be accurately tracked to the violating application?

More specifically, what infrastructure elements should be necessary for this large project and which are optional but very helpful. One such example I can think of is some form of a common logging infrastructure that allows for a developer or tester to easily browse through a log that contains numerous components for messages that might allude to the culprit program along with a "trail" of what happened before the issue occurred.  I'm thinking of something similar to Androids alogcat tool.

These necessary infrastructure elements should be language-agnostic.

While these elements should be understood by all engineers on the team in question, which elements should be understood at great detail by the technical system engineers and what should the individual software engineers be responsible for adding to their tools to allow for such infrastructures to take hold?

Please feel free to ask for clarification if something does not make sense as I understand this question is very broad and needs some refinement.  I will refine as necessary from the answers and comments I receive.

Thanks for any help!

Update:

I am entering in a team that has maybe 5% of the code with Unit Tests and is just beginning to Instrument and Monitor.  Each software programmer (I say programmer and not engineer because not everyone on the team is an engineer) does not understand the basics of fail immediately and sanity checking.  Much of our software baseline is legacy code and is in the process of being ported over.  Unfortunately we don't have the man power to refactor a lot of the older components.  This is what led me to try and understand if there there are necessary infrastructure tools that can be used to detect and find bugs at the source in a much quicker fashion.  While I am not expecting a tool to magically do this, I was thinking there might be tools or configurations that allow for more easily finding bugs in a sea of components.

答:
Take a look at Microsoft's Enterprise Library, a set of libraries, of Application Blocks, designed to be exactly that - commonly shared tools that are used across a large application. They include components for Caching, Exception Handling, Logging, Data Access and other pieces that are orthogonal to your actual business logic - they should stay pretty much the same for all modules.

Naturally, the components of the Enterprise Library aren't an exhaustive list, but they're a good place to start and see what is common to many applications, and therefore should be extracted from the main code into shared utility libraries.


问:
How do people read big technical books?
I see that most of the good programmers have the habit of reading big books for learning about technology.

What does it really take to read technical books, apart from the real interest on the technology?

How can I improve my ability to read these books?

答:
Good books aren't necessarily big.  The O'Reilly books and the Pragmatic Programmer books tend to be pretty slim, and they are quite good.

I would suggest that you read some of the books recommended at https://stackoverflow.com/questions/1711/what-is-the-single-most-influential-book-every-programmer-should-read

But to answer your question:  It takes effort, in the same way as any other study subject does.  Careful reading, reflection, notes taking and doing the excercises.

Title: How do people read big technical books?


答:
When one program fails in a sea of multiple components and processes, what infrastructure elements are necessary to ensure that the problem can be accurately tracked to the violating application?

I would argue that you're asking the wrong question. It's good to prepare for the eventuality that things go wrong, but to depend on humans to do this in a large distributed systems defeats many of the benefits of the large distributed system...

Things to focus on instead:

Unit Tests - 80%+ of your code isn't going to interface with the IPC. Make sure it works (with various bad inputs, behaviors) and that eliminates many of the issues that can impact a particular application.

Monitoring - Trying to 'trace back' issues is going to end in frustration, even if your logging infrastructure is working properly. Having a good monitoring system setup to identify issues early and specific to the process lets you see immediately what had issues.

Design for unambiguity - Either fail or work. Don't pass along bad data; don't 'kinda work'. If an app identifies that its IPC pair has disconnected or done something odd, fail right there. It's then immediately apparent who is responsible.

Title: Necessary Infrastructure for large project with many components communicating through IPCs

Body:
I have a fairly in depth question which probably doesn't have an exact answer.

As a software engineer, I am usually tasked with working on a program or project with minimal understanding of how other components or programs in the project interact with each other.  When one program fails in a sea of multiple components and processes, what infrastructure elements are necessary to ensure that the problem can be accurately tracked to the violating application?

More specifically, what infrastructure elements should be necessary for this large project and which are optional but very helpful. One such example I can think of is some form of a common logging infrastructure that allows for a developer or tester to easily browse through a log that contains numerous components for messages that might allude to the culprit program along with a "trail" of what happened before the issue occurred.  I'm thinking of something similar to Androids alogcat tool.

These necessary infrastructure elements should be language-agnostic.

While these elements should be understood by all engineers on the team in question, which elements should be understood at great detail by the technical system engineers and what should the individual software engineers be responsible for adding to their tools to allow for such infrastructures to take hold?

Please feel free to ask for clarification if something does not make sense as I understand this question is very broad and needs some refinement.  I will refine as necessary from the answers and comments I receive.

Thanks for any help!

Update:

I am entering in a team that has maybe 5% of the code with Unit Tests and is just beginning to Instrument and Monitor.  Each software programmer (I say programmer and not engineer because not everyone on the team is an engineer) does not understand the basics of fail immediately and sanity checking.  Much of our software baseline is legacy code and is in the process of being ported over.  Unfortunately we don't have the man power to refactor a lot of the older components.  This is what led me to try and understand if there there are necessary infrastructure tools that can be used to detect and find bugs at the source in a much quicker fashion.  While I am not expecting a tool to magically do this, I was thinking there might be tools or configurations that allow for more easily finding bugs in a sea of components.


问:
Does distributing non-GPLd assets with a GPL application violate the license?
This is somewhat related to my other question, but is actually different.

I would like to license a Windows Phone application under the GPL. All other Windows Phone Marketplace issues aside (I'll ask those on the forums), I'd like to include icons that ship with the SDK in my application. While this is common practice (documentation points to the icons' location), I'm not sure if I'd be forcing GPL on the icons (a move expressly forbidden by the Application Provider Agreement).

How is this usually handled in GPL or am I simply out of luck?

EDIT

The icons in question are copied into the project's source tree and distributed with the application.

The license that ships with the SDK states that  you may not:

modify or distribute the source code of any Distributable Code [incl. the icons] so that any part of it becomes subject to an Excluded License. An Excluded License is one that requires, as a condition of use, modification or distribution, that

the code be disclosed or distributed in source code form; or

others have the right to modify it.

My question now becomes: can I distribute the icons with my GPL'd application in any way (for example, by including provisions in my license text) that would not violate either the GPL or the SDK agreement?

答:
First off, you can't 'force' GPL on the icons - they aren't yours, so you can't decide their licensing. They remain someone else's property, no matter what you say. The worst you can do is release your program in violation of the GPL, and create legal headaches for yourself and for anyone who later tries to work with your code.

Second, remember that we're not lawyers, just opinionated people who've read some things.

What's the documentation specify as the license for the icons, or the SDK itself? I'm kind of presuming first off that it's OK to use them for building apps for Windows Phone, and second off that there's nothing in the SDK license that precludes using the GPL on applications.

Are the icons referenced from the SDK, or do you copy them into your project? If they are referenced from the SDK at build time, then they won't be part of your source code distribution. They'll probably be compiled into your app at build time, but there's probably several library stubs and chunks that are compiled in at build or link time, so I don't think the icons would be any different.


问:
Should we hire a new developer now, or wait until the code is refactored to make it suitable for a team environment?
I support and develop a large system that uses various technologies e.g. c++,.net,vb6 etc.  I am a sole developer.

I am debating whether now is the right time to approach my  manager (who is not a developer) to ask if another developer can be recruited.

I don't have any experience working in software teams.  I have always been a sole developer.

The concerns I have are:

There is still a lot to do.  Training another developer would take time and distract me from my duties.

The company does not invest heavily in tools e.g. source control

The code in this system needs to be refactored to introduce concepts such as interfaces, polymorphism etc, which are supported by methodologies such as Agile (I inherited the system about 12 months ago).  I am gradually trying to refactor the code.

I believe I have two options:

Approach my manager now

Wait until I have had time to refactor the code so it is more suitable for a team environment.

Which option is best? I am hoping to hear from other developers who have been in my situation.

答: Your new developer can help you refactor the code.  That will be his training (getting familiar with your software).  Hire someone that already knows interfaces, polymorphism and agile.  Implement a source control system yourself, if necessary.


答:
If you have an extra box, you can have source control. You can even have source control without an extra box, especially if you use a distributed source control system like mercurial or git.

The new developer can help you refactor your code. This is good for multiple reasons:

The developer learns the codebase as they refactor it

There's no need to go to the business and say, "Hey, invest in this time consuming process that won't net any additional business features."

Even agile code bases have bad code. Don't think you have to have good code now to be able to hire someone.

Title: Should we hire a new developer now, or wait until the code is refactored to make it suitable for a team environment?

Body:
I support and develop a large system that uses various technologies e.g. c++,.net,vb6 etc.  I am a sole developer.

I am debating whether now is the right time to approach my  manager (who is not a developer) to ask if another developer can be recruited.

I don't have any experience working in software teams.  I have always been a sole developer.

The concerns I have are:

There is still a lot to do.  Training another developer would take time and distract me from my duties.

The company does not invest heavily in tools e.g. source control

The code in this system needs to be refactored to introduce concepts such as interfaces, polymorphism etc, which are supported by methodologies such as Agile (I inherited the system about 12 months ago).  I am gradually trying to refactor the code.

I believe I have two options:

Approach my manager now

Wait until I have had time to refactor the code so it is more suitable for a team environment.

Which option is best? I am hoping to hear from other developers who have been in my situation.


答:
We have really BIG eyes.

All kidding aside, I'm one of the people who finds reading to be very difficult. If I'm working my way through a very large book, I try to read early in the morning, when I first wake up, when my mind is free of distractions. I find that I'm able to get engrossed much easier at that time of day and I retain more.

Then, there are books that are just so dry that they will be painful no matter the reading circumstances. I try to avoid them whenever possible, or find another book with the same information that is written in a different style. If reading a book is so painful that you can barely keep from putting it down, you are wasting your time because you probably won't retain much anyway.

Still, I much prefer getting information in smaller doses. My 'big books' are mostly for reference and aren't intended to be read cover to cover, unless you have an amazing attention span.

Additionally, though sort of digressing, I really enjoy it when people take time to write book reviews on their blog or personal web site. That helps me to find books that are best suited to me. So, if you love or hate a book, consider publishing a review. It will turn up to people who might be interested in whatever book you are discussing.

Title: How do people read big technical books?

Body:
I see that most of the good programmers have the habit of reading big books for learning about technology.

What does it really take to read technical books, apart from the real interest on the technology?

How can I improve my ability to read these books?