问:
Static vs. dynamic memory allocation - lots of constant objects, only small part of them used at runtime
Here are two options:

Option 1:

```<code>enum QuizCategory {

    CATEGORY_1(new MyCollection<Question>()
                    .add(Question.QUESTION_A)
                    .add(Question.QUESTION_B)
                    .add...),

    CATEGORY_2(new MyCollection<Question>()
                    .add(Question.QUESTION_B)
                    .add(Question.QUESTION_C)
                    .add...),

    ...

    ;

    public MyCollection<Question> collection;

    private QuizCategory(MyCollection<Question> collection) {
        this.collection = collection;
    }

    public Question getRandom() {
        return collection.getRandomQuestion();
    }

}
</code>```

Option 2:

```<code>enum QuizCategory2 {

    CATEGORY_1 {

        @Override
        protected MyCollection<Question> populateWithQuestions() {
            return new MyCollection<Question>()
                                .add(Question.QUESTION_A)
                                .add(Question.QUESTION_B)
                                .add...;
        }
    },

    CATEGORY_2 {

        @Override
        protected MyCollection<Question> populateWithQuestions() {
            return new MyCollection<Question>()
                                .add(Question.QUESTION_B)
                                .add(Question.QUESTION_C)
                                .add...;
        }
    };

    public Question getRandom() {
        MyCollection<Question> collection = populateWithQuestions();
        return collection.getRandomQuestion();
    }

    protected abstract MyCollection<Question> populateWithQuestions();

}
</code>```

There will be around 1000 categories, each containing 10 - 300 questions (100 on average). At runtime typically only 10 categories and 30 questions will be used. Each question is itself an enum constant (with its fields and methods).

I'm trying to decide between those two options in the mobile application context. I haven't done any measurements since I have yet to write the questions and would like to gather more information before committing to one or another option. As far as I understand:

(a) Option 1 will perform better since there will be no need to populate the collection and then garbage-collect the questions;
(b) Option 1 will require extra memory: 1000 categories x 100 questions x 4 bytes for each reference = 400 Kb, which is not significant.

So I'm leaning to Option 1, but just wondered if I'm correct in my assumptions and not missing something important? Perhaps someone has faced a similar dilemma? Or perhaps it doesn't actually matter that much?

答:
Is there any particular reason you are loading data you know won't be used?  Try to constrain those queries/list loaders to only data you intend to use.

Instead of loading the whole data set first and then filtering, apply the random selection function against an index/key for a dataset, and then load only those particular records with a matching index/key.

If you don't have an index or key to query against, and an index/key can't be added in a long-lived fashion, perform an in-memory dataset transform and add a synthetic or fake key to each, then cache that transformed and keyed data.  From there on out, refer to the cached data using the added key, until the cache is invalid.


问:
how to follow python polymorphism standards with math functions
So I am reading up on python in Mark Lutz's wonderful LEARNING PYTHON book. Mark makes a big deal about how part of the python development philosophy is polymorphism and that functions and code should rely on polymorphism and not do much type checking. However, I do a lot of math type programming and so the idea of polymorphism does not really seem to apply--I don't want to try and run a regression on a string or something. So I was wondering if there is something I am missing here. What are the applications of polymorphism when I am writing functions for math--or is type checking philosophically okay in this case. </p>

答:
The point of polymorphism is that if you write a regression function for, say, Python's built-in numeric types, someone else can use it with NumPy or some other numeric library and not think about it. You don't even have to know about the existence of such libraries to be confident in writing your code correctly. Just use the natural language structures, and leave it to whoever makes the next big numeric library to ensure their numeric types can be operated on normally, their lists and sequences can be iterated over by for loops, etc.

Breaking polymorphism to defend against sloppy coding or to enforce a certain use of your library doesn't really gain you anything. But it might restrict future users of your code for no good reason.

I was wondering if there is something I am missing here

Not missing anything, but maybe overcomplicating things. You write a function that takes the arguments you want, and document it accordingly. That's it.

I don't want to try and run a regression on a string or something

Neither you nor anyone else should be passing a string into a regression function. Who would do that? Code should not be doing anything by accident — if it is, you're just programming by guesswork, which is doomed to failure in any language.

is type checking philosophically okay in this case.

The answer to this is almost always no.

The general philosophy of Python is that you assume the other people using your code are going to be sensible about it — because if they're not, there's really nothing that will help. No type system, no checks, no built-in language features can defend against someone mashing their face on the keyboard and calling your function with the result.

The overarching reason is this: Python is designed for writing concise, expressive, readable code with a minimum of boilerplate. If you manually add in all the boilerplate yourself, you're defeating the point. So just relax and use the language to express your ideas as neatly as possible :)

ContentLicense: CC BY-SA 3.0


问:
What to call objects that may delete cached data to meet memory constraints?
I'm developing some cross-platform software which is intended to run on mobile devices. Both iOS and Android provide low memory warnings. I plan to make a wrapper class that will free cached resources (like textures) when low memory warnings are issued (assuming the resource is not in use). If the resource returns to use, it'll re-cache it, etc... I'm trying to think of what this is called. In .Net, it's similar to a "weak reference" but that only really makes sense when dealing with garbage collection, and since I'm using c++ and shared_ptr, a weak reference already has a meaning which is distinct from the one I'm thinking of. There's also the difference that this class will be able to rebuild the cache when needed. What is this pattern/whatever is called?

Edit: Feel free to recommend tags for this question.

答:
It is called a ... Cache.  :-). Seriously, intelligently flushing it is just as important to a good cache implementation than intelligently filling it.

But your example is not really a cache, since it is not optional. A texture MUST be loaded before it can be rendered. (I would use the terms load and unload in this case).

So I imagine a class:

That is asked to make sure a texture image is loaded, every time it is used.

That loads the image from the correct storage (eg. A certain resource folder)

To keep track of texture usage patterns.

which can be asked to unload textures, based on these usage patterns.

Then that would be called a TextureLoader to me (short for TextureLoaderUnloader)


答:
You are certainly allowed to do input checking. In some languages, you might check input arguments to make sure they are the appropriate type (i.e., they are of the right class). However, in Python, it is often preferred to check whether the input exposes the necessary capabilities. This is sometimes called Duck Typing, after the maxim that if something walks like a duck and quacks like a duck, it's probably a duck--or at least something that a duck processing function should be able to handle.

To give you a more concrete example, suppose you are writing a function that calculates the mean of a list of numbers. Instead of checking whether the parameter is, in fact, a list, check whether it is iterable (or even better, just try to iterate over it and throw an exception if not). This lets your function work for lists, tuples, sets, files, values from databases, and even as-of-yet-uninvented objects that store collections of numbers. On the other hand, you are certainly allowed, and even encouraged, to do domain-specific checking. If your code receives mathematically nonsensical values (e.g., negative probabilities), it probably should throw an exception or somehow signal an error.

Overall, The Python philosophy seems to be "check as much as you need to, but no more. If the caller does something strange, it's at least partly their problem (like regressing on a string). That said, these are guidelines, not hard and fast rules, so use common sense.

Title: how to follow python polymorphism standards with math functions

Body: So I am reading up on python in Mark Lutz's wonderful LEARNING PYTHON book. Mark makes a big deal about how part of the python development philosophy is polymorphism and that functions and code should rely on polymorphism and not do much type checking. However, I do a lot of math type programming and so the idea of polymorphism does not really seem to apply--I don't want to try and run a regression on a string or something. So I was wondering if there is something I am missing here. What are the applications of polymorphism when I am writing functions for math--or is type checking philosophically okay in this case. </p>


答:
One term used for this is "Scavenger", though I've mostly seen it in the context of Microsoft technologies. See here in the ASP.NET Caching Overview page:

Scavenging

Scavenging is the process of deleting items from the cache when memory is scarce. Items are removed when they have not been accessed in some time or when items are marked as low priority when they are added to the cache.

And here's a definition from Microsoft's Caching in WinINet page:

The cache scavenger periodically cleans items from the cache. If an item is added to the cache and the cache is full, the item is added to the cache and the cache scavenger is scheduled.

So if you have an object whose task it is to free cached items when resources are low, it would be a Scavenger. You might have several scavenging strategies - Biggest First, Earliest First, etc., each implemented by a different Scavenger.

Title: What to call objects that may delete cached data to meet memory constraints?

Body:
I'm developing some cross-platform software which is intended to run on mobile devices. Both iOS and Android provide low memory warnings. I plan to make a wrapper class that will free cached resources (like textures) when low memory warnings are issued (assuming the resource is not in use). If the resource returns to use, it'll re-cache it, etc... I'm trying to think of what this is called. In .Net, it's similar to a "weak reference" but that only really makes sense when dealing with garbage collection, and since I'm using c++ and shared_ptr, a weak reference already has a meaning which is distinct from the one I'm thinking of. There's also the difference that this class will be able to rebuild the cache when needed. What is this pattern/whatever is called?

Edit: Feel free to recommend tags for this question.


问:
What's the best way to create a static utility class in python? Is using metaclasses code smell?
Ok so I need to create a bunch of utility classes in python. Normally I would just use a simple module for this but I need to be able to inherit in order to share common code between them. The common code needs to reference the state of the module using it so simple imports wouldn't work well. I don't like singletons, and classes that use the classmethod decorator do not have proper support for python properties.

One pattern I see used a lot is creating an internal python class prefixed with an underscore and creating a single instance which is then explicitly imported or set as the  module itself. This is also used by fabric to create a common environment object (fabric.api.env).

I've realized another way to accomplish this would be with metaclasses. For example:

```<code>#util.py
class MetaFooBase(type):
    @property
    def file_path(cls):
        raise NotImplementedError

    def inherited_method(cls):
        print cls.file_path

#foo.py
from util import *
import env

class MetaFoo(MetaFooBase):
    @property
    def file_path(cls):
        return env.base_path + "relative/path"

    def another_class_method(cls):
        pass

class Foo(object): __metaclass__ = MetaFoo

#client.py
from foo import Foo

file_path = Foo.file_path
</code>```

I like this approach better than the first pattern for a few reasons:

First, instantiating Foo would be meaningless as it has no attributes or methods, which insures this class acts like a true single interface utility, unlike the first pattern which relies on the underscore convention to dissuade client code from creating more instances of the internal class.

Second, sub-classing MetaFoo in a different module wouldn't be as awkward because I wouldn't be importing a class with an underscore which is inherently going against its private naming convention.

Third, this seems to be the closest approximation to a static class that exists in python, as all the meta code applies only to the class and not to its instances. This is shown by the common convention of using cls instead of self in the class methods. As well, the base class inherits from type instead of object which would prevent users from trying to use it as a base for other non-static classes. It's implementation as a static class is also apparent when using it by the naming convention Foo, as opposed to foo, which denotes a static class method is being used.

As much as I think this is a good fit, I feel that others might feel its not pythonic because its not a sanctioned use for metaclasses which should be avoided 99% of the time. I also find most python devs tend to shy away from metaclasses which might affect code reuse/maintainability. Is this code considered code smell in the python community? I ask because I'm creating a pypi package, and would like to do everything I can to increase adoption.

答:
I think it is a bad idea.

Its a very odd use of meta classes which will contribute to confusion of anyone trying to follow your code. In order to make it worth using this technique, you need the advantages to outweigh that confusion.

As for your reasons:

Creating an accesible Foo class that can be instantiated, but doesn't actually work like a typical object is much more prone to misuse than an internal class that you know you aren't supposed to call.

Don't name the base class with an underscore. If its intended to be inherited by other modules, it shouldn't be named that way.

There already is a method to implement static classes in python, staticmethod or classmethod. As noted, its not hard to implement a staticproperty

In general, there is a theme in your approach of trying to prevent client code from doing the wrong thing. That's not a pythonic attitude. The pythonic approach is to trust your client code. If they want to do something crazy, let them. Don't try to prevent client from instantiating your internal object, there may well be a good reason to do that.

I'm a bit of a purist and think that you shouldn't be storing state like this. So the very fact that you are asking the question means you're already doing it wrong. If you have state, I think it should be in an object. I think storing any sort of state at the module level should be avoided. I think its only a good idea to implement something like this if you have no changing state.

To close off, I'd like to plug the Code Review Stack Exchange, where I'm a moderator. If you post your code there you can get suggestions on how to make it better. I think you may get some helpful suggestions for how to restructure your code to avoid this question from even coming up.


答:
If the differences of two approaches are neither significant in runtime nor in performance (and significant means: from a users point of view, not from an academic point of view), then choose the approach which is easier to implement and to maintain.

Actually, seeing only that small part of your application I cannot tell you which of those two approaches is easier to handle at the long term, but the first impression is that "Option 1" is not just faster, but also seems to need less code (at a first glance). The questions you should ask yourself before making the final decision here are:

does your memory extrapolation stick valid for the next months (or years) when the program runs in production?

does the list of categories / questions never change during the time your programs runs? (if it can change, there will be obviously a need to reload that objects from somewhere at some point in time)

what other operations (besides from getRandom) working with collection will be needed in your program? Are those operations easier to be implemented when you preload everything?

Furthermore, providing an accessor getCollection may be good idea, then you can hide the loading / reloading / lazy loading in the implementation of that accessor and change it later, when needed. So make it

```<code>    public Question getRandom() {
        return getCollection().getRandomQuestion();
    }
</code>```

then you can change your initial decision later more easily.

Title: Static vs. dynamic memory allocation - lots of constant objects, only small part of them used at runtime

Body:
Here are two options:

Option 1:

```<code>enum QuizCategory {

    CATEGORY_1(new MyCollection<Question>()
                    .add(Question.QUESTION_A)
                    .add(Question.QUESTION_B)
                    .add...),

    CATEGORY_2(new MyCollection<Question>()
                    .add(Question.QUESTION_B)
                    .add(Question.QUESTION_C)
                    .add...),

    ...

    ;

    public MyCollection<Question> collection;

    private QuizCategory(MyCollection<Question> collection) {
        this.collection = collection;
    }

    public Question getRandom() {
        return collection.getRandomQuestion();
    }

}
</code>```

Option 2:

```<code>enum QuizCategory2 {

    CATEGORY_1 {

        @Override
        protected MyCollection<Question> populateWithQuestions() {
            return new MyCollection<Question>()
                                .add(Question.QUESTION_A)
                                .add(Question.QUESTION_B)
                                .add...;
        }
    },

    CATEGORY_2 {

        @Override
        protected MyCollection<Question> populateWithQuestions() {
            return new MyCollection<Question>()
                                .add(Question.QUESTION_B)
                                .add(Question.QUESTION_C)
                                .add...;
        }
    };

    public Question getRandom() {
        MyCollection<Question> collection = populateWithQuestions();
        return collection.getRandomQuestion();
    }

    protected abstract MyCollection<Question> populateWithQuestions();

}
</code>```

There will be around 1000 categories, each containing 10 - 300 questions (100 on average). At runtime typically only 10 categories and 30 questions will be used. Each question is itself an enum constant (with its fields and methods).

I'm trying to decide between those two options in the mobile application context. I haven't done any measurements since I have yet to write the questions and would like to gather more information before committing to one or another option. As far as I understand:

(a) Option 1 will perform better since there will be no need to populate the collection and then garbage-collect the questions;
(b) Option 1 will require extra memory: 1000 categories x 100 questions x 4 bytes for each reference = 400 Kb, which is not significant.

So I'm leaning to Option 1, but just wondered if I'm correct in my assumptions and not missing something important? Perhaps someone has faced a similar dilemma? Or perhaps it doesn't actually matter that much?

Body:
If the differences of two approaches are neither significant in runtime nor in performance (and significant means: from a users point of view, not from an academic point of view), then choose the approach which is easier to implement and to maintain.

Actually, seeing only that small part of your application I cannot tell you which of those two approaches is easier to handle at the long term, but the first impression is that "Option 1" is not just faster, but also seems to need less code (at a first glance). The questions you should ask yourself before making the final decision here are:

does your memory extrapolation stick valid for the next months (or years) when the program runs in production?

does the list of categories / questions never change during the time your programs runs? (if it can change, there will be obviously a need to reload that objects from somewhere at some point in time)

what other operations (besides from getRandom) working with collection will be needed in your program? Are those operations easier to be implemented when you preload everything?

Furthermore, providing an accessor getCollection may be good idea, then you can hide the loading / reloading / lazy loading in the implementation of that accessor and change it later, when needed. So make it

    public Question getRandom() {
        return getCollection().getRandomQuestion();
    }

then you can change your initial decision later more easily.


问:
Naming conventions used for variables and functions in C
While coding a large project in C I came upon a problem. If I keep on writing more code then there will be a time when it will be difficult for me to organize the code. I mean that the naming for functions and variables for different parts of the program may seem to be mixed up.

So I was thinking whether there are useful naming conventions that I can use for C variables and functions?

Most languages suggest a naming convention. But for C the only thing I have read so far is the names should be descriptive for code readability.

EDIT:

Examples of some examples of suggested naming conventions:

Python's PEP 8

Java Tutorial

I read some more naming conventions for java somewhere but couldn't remember where.

答:
Most of the answers are good, but I want to say some things about naming conventions for libaries and included files, similar to using namespaces in other languages like C++ or Java:

If you build a library, find a common prefix for your exported symbols, i.e. global functions, typedefs and variables. This will prevent clashes with other libraries and identify the functions as coming from yours. This is a little bit of apps Hungarian notations.

Maybe go even further and group your exported symbols: libcurl uses curl_* for global symbols, curl_easy_*, curl_multi_*, and curl_share_* for the different interfaces. So in addition to using curl_* for all functions, they have added another level of "namespaces" for the different interfaces: calling a curl_easy_* function on a curl_multi_* handle now looks wrong, see the function names at http://curl.haxx.se/libcurl/c/

Keeping the rules for exported symbols, you should use those for static functions in #includeed files: Try to find a common prefix for these functions. Maybe you have static string utility functions in a file called "my_string"? Prefix all those functions with my_string_*.


问:
How advanced are author-recognition methods?
From a written text by an author if a computer program analyses the text, how much can a computer program tell today about the author of some (long enough to be statistically significant) texts?

Can the computer program even tell with "certainty" whether a man or a woman wrote this text based solely on the contents of the text and not an investigation such as ip numbers etc?

I'm interested to know if there are algorithms in use for instance to automatically know whether an author was male or female or similar characteristics of an author that a computer program can decide based on analyses of the written text by an author.

It could be useful to know before you read a message what a computer analyses says about the author, do you agree? If I for instance get a longer message from my wife that she has had an accident in Nigeria and the computer program says that with 99 % probability the message was written by a male author in his sixties of non-caucasian origin or likewise, or by somebody who is not my wife, then the computer program could help me investigate why a certain message differs in characteristics.

There can also be other uses for instance just detecting outliers in a geographically or demographically bounded larger data set.

Scam detection is the obvious use I'm thinking of but there could also be other uses. Are there already such programs that analyse a written text to tell something about the author based on word choice, use of pronouns, unusual language usage, or likewise?

答:
Yes, there are, and no, they don't work very well.

Deducing information about the author from a text is sub-discipline of natural language processing - most NLP applications are about extracting information about the content of a text rather than the author, but the goals, methods and state of the art are actually rather similar (currently this favors things suchas n=-gram counts, maximum-entropy classificators etc.). In the end, understanding a text and understanding its author are both small parts of the old computer science dream, artificial intelligence. Like most of the problems in AI, both have turned out to be much, much harder than expected, very much dependent on domains, circumstances and processing power, and progressing only slowly and arduously.

That said, there are established methods for tasks such as "sentiment analysis" (deciding whether a text, e.g. a customer review, is positive or negative), summarizing (extracting the key message from a passage of text) or question answering that work tolerably well under controlled conditions. Author detection is harder than either of these; you can sometimes detect a particular writer by characteristic phrases, constructions, topics or opinions, but often you can't, and the same indicators that work quite well for one author can be totally useless for others. That is even before considering that people can change their writing style deliberately, specifically to defeat being unmasked. In fact, if you had a reliable algorithm for detecting authors, this would be a very big help to someone trying to escape detection, since he would only have to keep paraphrasing until the algorithm no longer identifies him!

This is a general problem with text processing to defeat human intentions: the results can be used by both sides, which often annihilates any progress that the scientists make. For instance, many teachers use online plagiarism detection services, but that only works because the teachers put more effort into detecting plagiarism than the students put into plagiarizing in the first place. If someone really wants to get away with submitting someone else's work, they just have to subscribe to the same services and try out which solutions will be detected and which won't.

So altogether, the field is huge, frustrating but fascinating, and nowhere near ready for reliable use for what you have in mind.


问:
Agile bug fixing - what's the preferred process for testing?
When a bug is fixed, the dev set its status to "resolved" and the bug is reassigned back to the person that created it. In our case this is usually the product owner - we don't have dedicated testers.

But what's a good process for controlling how/when the PO tests the software? Should he be given the latest build after each bug is resolved/checked-in? Or what about every morning? Or should he only receive a build at (or close to) the end of the iteration, to include all of that iteration's new functionality and bug fixes?

We are using TFS by the way.

答:
It seems to me that you are mixing up two kinds of testing.

Functional Testing is when you check for regressions or that a bug is fixed. This can be done via automated tests or a manual test by a developer. E.g. "does functionality X work?"

Acceptance Testing is when the PO verifies that a story or a unit of work implements what was asked. This can also be automated but it's semantically separate from the previous. E.g. "does the software now allow me to achieve Y?"

A lot of people seem to miss the difference between the two, however in my experience there's a huge gap and mastering what these two forms of testing are is fundamental to a successful project.

POs don't test software. They perform a cursory check to make sure that the product being developed is going in the correct direction. This means that there has to be someone else making sure the software is tested. This role, though, should in turn completely refrain from judging where the product is going in terms of features.

Projects that mix up the two things tend to fail:

QAs will become unelected mini-POs and fail at that

POs won't test your software and if they do, they'll lose the ability to judge from a distance.


问:
Uses of persistent data structures in non-functional languages
Languages that are purely functional or near-purely functional benefit from persistent data structures because they are immutable and fit well with the stateless style of functional programming.

But from time to time we see libraries of persistent data structures for (state-based, OOP) languages like Java.  A claim often heard in favor of persistent data structures is that because they are immutable, they are thread-safe.

However, the reason that persistent data structures are thread-safe is that if one thread were to "add" an element to a persistent collection, the operation returns a new collection like the original but with the element added.  Other threads therefore see the original collection.  The two collections share a lot of internal state, of course -- that's why these persistent structures are efficient.

But since different threads see different states of data, it would seem that persistent data structures are not in themselves sufficient to handle scenarios where one thread makes a change that is visible to other threads.  For this, it seems we must use devices such as atoms, references, software transactional memory, or even classic locks and synchronization mechanisms.

Why then, is the immutability of PDSs touted as something beneficial for "thread safety"?  Are there any real examples where PDSs help in synchronization, or solving concurrency problems?  Or are PDSs simply a way to provide a stateless interface to an object in support of a functional programming style?

答:
One can imagine a data structure which would be persistent but mutable. For example, you could take a linked list, represented by a pointer to the first node, and a prepend-operation which would return a new list, consisting of a new head node plus the previous list. Since you still have the reference to the previous head, you can access and modify this list, which has meanwhile become also embedded inside the new list. While possible, such a paradigm doesn't offer the benefits of persistent and immutable data structures, e.g. it is certainly not thread safe by default. However, it may have its uses as long as the developer knows what they're doing, e.g. for space efficiency. Also note that while the structure may be mutable at the language level in that nothing prevents the code from modifying it, it may in practice be used as if it were immutable: the application logic may by convention not mutate the state even though theoretically it could.

So long story short, without immutability (enforced by the language or by convention), persistence od data structures loses some of its benefits (thread safety) but not others (space efficiency for some scenarios).

As for examples from non-functional languages, Java's String.substring() uses what I would call a persistent data structure. The String is represented by an array of characters plus the start and end offsets of the range of the array which is actually used. When a substring is created, the new object re-uses the same character array, only with modified start and end offsets. Since String is immutable, it is (with respect to the substring() operation, not others) an immutable persistent data structure.

The immutability of data structures is the part relevant to thread safety. Their persistence (re-use of existing chunks when a new structure is created) is relevant to efficiency when working with such collections. Since they are immutable, an operation like adding an item doesn't modify the existing structure but returns a new one, with the additional element appended. If each time the whole structure was copied, starting with an empty collection and adding 1000 elements one by one in order to end up with a 1000-element collection, would create temporary objects with 0+1+2+...+999 = 500000 elements total which would be a huge waste. With persistent data structures, this can be avoided as the 1-element collection is re-used in the 2-element one, which is re-used in the 3-element one and so on, so that in the end no garbage nodes are allocated - each one is at the end used in the final state of the data structure.


问:
Sync csv file using nodejs
There is a remote csv file that gets updated every second or so.

I need to download it(on a Windows machine) ONCE and always sync local file with the remote one.

Obviously, downloading the whole file every time is not an option. I need to download only the changes.(something like rsync, rdiff-backup)

I searched quite a bit but could not find how I can do this.

I am sort of new to nodejs and am using this app as an opportunity to expand my nodejs skills. Also, I am planning to use nodejs and to package it using node-webkit(https://github.com/rogerwang/node-webkit)

答:
As the comments say, this is impossible without the remote server providing some sort of native diff.

I see two options

Download the whole file and do a diff yourself - Potentially expensive, but works out of the box without requiring the remote server to add diff features.

Add support for an update stream on the remote server - This would allow you to download the file once and then somehow subscribe to updates. The best way to do this entirely depends on the protocol which you have not specified. It could be as easy as running rsync or as complicated as a separate CSV file that has entries specifying the row, column, and the new value since the last fully generated main CSV file.

Title: Sync csv file using nodejs


问:
Algorithms for Data Redundancy and Failover for distributed storage system?
I'm building a distributed storage system that works with different storage sizes. For instance, my storage devices have sizes of 50GB, 70GB, 150GB, 250GB, 1000GB, 5 storage systems in one system. My application will store any files to the storage system.

Question: How can I build a distributed storage with the idea of data redundancy and fail-over to store documents, videos, any type of files at the same time ensuring that should one of any storage devices fail, there would be another copy of these files on another storage device. However, the concern is, 50GB of storage can only store this maximum number of files as compared to 70GB, 150GB etc. With one storage in mind, bringing 5 storage systems like a cloud storage, is there any logical way to distribute or store the files through my application?

How do I ensure data redundancy through different storage sizes?
Is there any algorithm to collate multiple blob files into a single file archive?
What is the best solution for one cloud storage with multiple different storage sizes?

I open this topic with the objective of discussing the best way to implement this idea, assuming simplicity, what are the issues of this implementation, performance measurements and discussion of the limitations.

答:
There are two classic, NP-hard problems that relate to your question.

Bin Packing Problem

Knapsack Problem

As they are both NP-hard, there really isn't a "perfect" a priori algorithm to handle solving the problem.  So that answers your 3rd question - there isn't a "one best system" to solve this.  On the other hand, those articles provide quite a few approximations you can use to "cheat" the problem and come up with a reasonable solution.

From a higher level point of view, consider having some sort of meta-data storage layer (aka a database) in order to keep track of what files are in your backup system as well where the primary and backup copies are located.  That will help with your first question.  Having redundant copies is simply part of your business logic and you can write rules to verify that logic is met or to take corrective action by making copies.

For your second question, the knapsack problem directly applies -- multiple blobs in a single file is a pretty simple restatement of the classic problem.


问:
Does data size in TCP/UDP make a difference on transmission time
While discussing the development of a network component for our game engine, a member of our team suggested that transmitting either 500 bytes or 1k of data using UDP makes no difference from performance perspective of the system (the time it takes to transmit the data).

He actually said that as long as you don't cross the MTU size, the size of the transmitted data doesn't really matter as it's all the same.

Is that true for UDP? what about TCP?

That sounds just plain wrong to me, but i am not a network expert.

*I've been reading about other companies' game networking architectures, and it seems they're all trying to keep transmitted data to a minimum, making my colleague's claims seem even more unreasonable.

答:
It all depends on what you consider to be the "transmission time". If you take into account the time elapsed between the calls to send() and recv(), this depends on several factors, and packet size might not be so relevant to the calculation. In that case you could, in some (but I repeat, not all!) cases say that whatever you do - TCP, UDP, big or small packets - little changes.

Otherwise, it actually could be beneficial in some scenarios to have smaller packets in order to decrease total transmission time. For example: if you have a high error probability, and you need to retransmit, and you use very large packets, then very soon the error retransmission time begins to dominate the calculation. Increasing packet size will increase the cost and probability of an error, thus increasing the transmission time. In the extreme case of a single packet, the probability of that one packet to be damaged approaches unity, and you may need to retransmit one thousand times. Time increase is now 100000%. With packets one thousand times smaller, ten of them are damaged, you retransmit only those, and overall time is increased by as little as one percent (depending on retransmission protocol).

You also have to consider the overhead. Every packet of B payload bytes you send will have its O bytes of overhead. If we simplify and assume that packet physical transmission time is a linear function of its size (it is not), to transmit N bytes you need to transmit N/B frames, which means (N/B)*(B+O) = N(1 + O/B) bytes, and kN(1+O/B) transmission time. As you can see, the smaller is B, the longer the time. A hypothetical packet of size zero would mean no payload data transmission at all, and therefore infinite time, the exact opposite of the above scenario.

Only at the most abstract level - no overheads and constant-time transmission - yes, transmission time is k(N/B)*B, which is kN and does not depend on B. And, again yes, the first "accident" this model encounters is probably a discontinuity on hitting MTU size.

In general, however, that is not a faithful representation of what really is taking place (think overhead. Think errors. Think lower-layer overheads, plural... and this maybe on a route of several hops), and packet size might matter a lot.

Often, you will have to experiment in order to find the actual best value, because simple heuristics ("the smaller the better", "the bigger the better", etc.) will not yield the best possible result.

A simulated test

I'm here on a XP laptop, so netstat -snap tcp gives me some data to work with re. error rate. I haven't statistics on packet sizes, so I'm assuming they're all around 1500 bytes even if of course they aren't.

I assume that my stats give actual error rate and that a packet is dead if a byte is dead, so that error probability at byte level, Pb, is related to packet error probability Pp by the relation "for a packet to succeed, all bytes must come through intact", i.e.,

```<code>1 - Pp = (1 - Pb)^PacketSize
</code>```

This lets me derive Pb as 1-exp(ln(1 - Pp)/PacketSize). At that point I can recalculate the packet error probability for any other PacketSize, and from that, determine how long it will take to transmit the same payload.

In theory I could claim being right, for it turns out that size matters. But that's nitpicking - I was actually wrong, for in this scenario at least, size matters very very little:

So, packet size may still influence application design, but certainly not because of data transmission times.

Body:
It all depends on what you consider to be the "transmission time". If you take into account the time elapsed between the calls to send() and recv(), this depends on several factors, and packet size might not be so relevant to the calculation. In that case you could, in some (but I repeat, not all!) cases say that whatever you do - TCP, UDP, big or small packets - little changes.

Otherwise, it actually could be beneficial in some scenarios to have smaller packets in order to decrease total transmission time. For example: if you have a high error probability, and you need to retransmit, and you use very large packets, then very soon the error retransmission time begins to dominate the calculation. Increasing packet size will increase the cost and probability of an error, thus increasing the transmission time. In the extreme case of a single packet, the probability of that one packet to be damaged approaches unity, and you may need to retransmit one thousand times. Time increase is now 100000%. With packets one thousand times smaller, ten of them are damaged, you retransmit only those, and overall time is increased by as little as one percent (depending on retransmission protocol).

You also have to consider the overhead. Every packet of B payload bytes you send will have its O bytes of overhead. If we simplify and assume that packet physical transmission time is a linear function of its size (it is not), to transmit N bytes you need to transmit N/B frames, which means (N/B)*(B+O) = N(1 + O/B) bytes, and kN(1+O/B) transmission time. As you can see, the smaller is B, the longer the time. A hypothetical packet of size zero would mean no payload data transmission at all, and therefore infinite time, the exact opposite of the above scenario.

Only at the most abstract level - no overheads and constant-time transmission - yes, transmission time is k(N/B)*B, which is kN and does not depend on B. And, again yes, the first "accident" this model encounters is probably a discontinuity on hitting MTU size.

In general, however, that is not a faithful representation of what really is taking place (think overhead. Think errors. Think lower-layer overheads, plural... and this maybe on a route of several hops), and packet size might matter a lot.

Often, you will have to experiment in order to find the actual best value, because simple heuristics ("the smaller the better", "the bigger the better", etc.) will not yield the best possible result.

A simulated test

I'm here on a XP laptop, so netstat -snap tcp gives me some data to work with re. error rate. I haven't statistics on packet sizes, so I'm assuming they're all around 1500 bytes even if of course they aren't.

I assume that my stats give actual error rate and that a packet is dead if a byte is dead, so that error probability at byte level, Pb, is related to packet error probability Pp by the relation "for a packet to succeed, all bytes must come through intact", i.e.,

1 - Pp = (1 - Pb)^PacketSize

This lets me derive Pb as 1-exp(ln(1 - Pp)/PacketSize). At that point I can recalculate the packet error probability for any other PacketSize, and from that, determine how long it will take to transmit the same payload.

In theory I could claim being right, for it turns out that size matters. But that's nitpicking - I was actually wrong, for in this scenario at least, size matters very very little:

So, packet size may still influence application design, but certainly not because of data transmission times.


答:
Yes, size matters.  Your co-worker's argument is that any amount of traffic up to the MTU will take the same amount of time to transmit, and that simply isn't true.

Forget for a minute that you have any protocols at all, just a pipe where bits go in one end and (hopefully) make out out the other.  If the pipe can transfer 8,000 bits per second, 500 bytes (4,000 bits) will take half a second and 1,000 bytes (8,000 bits) will take a full second.  This holds true for pretty much every kind of interface because they're all serial.  Some interfaces are partially parallel in that they may let you transfer more than one bit per clock cycle, but once you've exceeded that limit, you're serially transferring a stream of whatever those units are.  So there's one basic limit right there.

When you add protocols, which are necessary because pipes aren't always reliable or you have lots of pipes that form a network, you add information that tell the protocol implementation what to do with the packet.  That information takes up space (and therefore transmission time) that you can no longer use for data and is required overhead, because you can't run the protocol without it.

Your example is a UDP/IP* packet, which contains your payload, an 8-byte header added by the UDP protocol and a 20-byte header added by the IP protocol.  If you send 500-byte datagrams, the 28-byte overhead imposed by the protocols will make up 5.3% of the 528-byte packet that has to be sent.  Increase the payload to 1,000 bytes and you're sending more of your own data in each 1,028-byte datagram for an overhead rate of 2.7%.  If you had a transmission medium that had a large MTU and could swallow a 10,028-byte packet, overhead would shrink even further to 0.3%.

TCP, being a more complex protocol than UDP, has a 20-byte header, which makes the total overhead 40 bytes when run over IP.  For your 500- and 1,000-byte payload examples, overhead rates would be 7.4% and 3.8%.

Sending those TCP/IP or UDP/IP packets over Ethernet adds 14 bytes of Ethernet's own header, four bytes of its trailer and 12 bytes of idle time between frames for a total of 30 bytes.  As it is with the protocols above, larger payloads mean fewer frames and fewer frames mean less time spent on overhead.

There are a number of reasons why you don't just send arbitrarily-large packets to keep the overhead down.  The biggest of those reasons is that you don't want to have to retransmit a whole megabyte of data because you lost six bytes somewhere in the middle.  Unless you know your transmission medium is extremely reliable, it's much better to incur the overhead so you only have to re-send a one-kilobyte frame.

The real-world parallel to this would be using 53-foot trucks for shipping.  It's a lot cheaper to pack each one as tightly as possible than it is to hire drivers and buy gas for a lot of them to carry just a couple of things each.

*What is commonly called UDP is actually UDP over IP or UDP/IP.  UDP can actually be run on top of protocols other than IP, but IP is by far the usual case.

To address dbasnett's comment:  The point isn't that UDP is run on top of other protocols, it's that its place in the protocol layer cake means it can be.  UDP, being a transport-layer protocol, assumes host addressing has already been taken care of by the network layer.  This means you can run UDP (and just UDP, not UDP/IP) across pretty much anything if your only needs were to identify sending and receiving ports.  A serial link between two hosts (with no network layer, since addressing would be implicit) would work.  Ethernet frames would, too, if MAC addresses were sufficient to identify hosts.  I'm not saying anyone actually does this, but a properly-designed network stack will allow replacement of lower layers without the higher ones having to know or care.

Title: Does data size in TCP/UDP make a difference on transmission time

Body:
While discussing the development of a network component for our game engine, a member of our team suggested that transmitting either 500 bytes or 1k of data using UDP makes no difference from performance perspective of the system (the time it takes to transmit the data).

He actually said that as long as you don't cross the MTU size, the size of the transmitted data doesn't really matter as it's all the same.

Is that true for UDP? what about TCP?

That sounds just plain wrong to me, but i am not a network expert.

*I've been reading about other companies' game networking architectures, and it seems they're all trying to keep transmitted data to a minimum, making my colleague's claims seem even more unreasonable.

ContentLicense: CC BY-SA 4.0


答:
Other answers commented on the data size part already but for measuring differences there is another factor relevant:

TCP does not only send the data packages but before sending those it sends extra packages to create a connection, then confirms retrieval of packages and finally sends extra packages to close the connection.

Now the benefit of this is that you get a guarantee that all data was received completely and in the correct order. Building those guarantees in the application is often more complicated than using the proven TCP implementations in the system. Even though Google is currently working on QUIC, an protocol based on UDP to send Web content, I'd suggest using TCP if you need any form of guarantee and UDP only when it is ok that arbitrary packages get lost.


答:
Persistent/immutable data structures don't solve concurrency problems on their own, but they make solving them much easier.

Consider a thread T1 that passes a set S to another thread T2. If S is mutable, T1 has a problem: It loses control of what happens with S. Thread T2 can modify it, so T1 can't rely at all on content of S. And vice versa - T2 can't be sure that T1 doesn't modify S while T2 operates on it.

One solution is to add some kind of a contract to the communication of T1 and T2 so that only one of the threads is allowed to modify S. This is error prone and burdens both the design and implementation.

Another solution is that T1 or T2 clone the data structure (or both of them, if they aren't coordinated). However, if S isn't persistent, this is an expensive O(n) operation.

If you have a persistent data structure, you're free of this burden. You can pass a structure to another thread and you don't have to care what it does with it. Both threads have access to the original version and can do arbitrary operations on it - it doesn't influence what the other thread sees.

See also: persistent vs immutable data structure.

Title: Uses of persistent data structures in non-functional languages

Body:
Languages that are purely functional or near-purely functional benefit from persistent data structures because they are immutable and fit well with the stateless style of functional programming.

But from time to time we see libraries of persistent data structures for (state-based, OOP) languages like Java.  A claim often heard in favor of persistent data structures is that because they are immutable, they are thread-safe.

However, the reason that persistent data structures are thread-safe is that if one thread were to "add" an element to a persistent collection, the operation returns a new collection like the original but with the element added.  Other threads therefore see the original collection.  The two collections share a lot of internal state, of course -- that's why these persistent structures are efficient.

But since different threads see different states of data, it would seem that persistent data structures are not in themselves sufficient to handle scenarios where one thread makes a change that is visible to other threads.  For this, it seems we must use devices such as atoms, references, software transactional memory, or even classic locks and synchronization mechanisms.

Why then, is the immutability of PDSs touted as something beneficial for "thread safety"?  Are there any real examples where PDSs help in synchronization, or solving concurrency problems?  Or are PDSs simply a way to provide a stateless interface to an object in support of a functional programming style?


问:
Dependency Injection Confusion
I think I have a decent grasp of what Dependency Inversion principle (DIP) is, my confusion is more around dependency injection.

My understanding is the whole point of DI is to decouple parts of an application, to allow changes in one part without effecting another, assuming the interface does not change.

For examples sake, we have this

```<code>public class MyClass(IMyInterface interface)
{
   public MyClass
   {
      interface.DoSomething();
   }
}

public interface IMyInterface
{
    void DoSomething();
}
</code>```

How is this

```<code>var iocContainer = new UnityContainer();
iocContainer.Resolve<MyClass>();
</code>```

better practice than doing this

```<code>//if multiple implementations are possible, could use a factory here.

IMyInterface interface = new InterfaceImplementation();
var myClass  = new MyClass(interface);
</code>```

It may be I am missing a very important point, but I am failing to see what is gained. I am aware that using an IOC container I can easily handle an objects life cycle, which is a +1 but I don't think that is core to what IOC is about.

Edit

Here is an expanded example:

```<code>void Main()
{
    IRepository repository = new AddRepository();
    var maths = new PointlessMaths(repository,5);
    Console.WriteLine(maths.Result);
}

public interface IRepository
{
    int DoSomething(int id);
}

public class AddRepository : IRepository
{
    public int DoSomething(int id)
    {
        return id + 1;
    }
}

public class SubtractRepository : IRepository
{
    public int DoSomething(int id)
    {
        return id - 1;
    }
}

public class PointlessMaths
{
    public int Result {get;set;}
    public PointlessMaths(IRepository repository, int id)
    {
        Result = repository.DoSomething(id);
    }
}
</code>```

答: In your example, you haven't gained anything. Your intuition is correct. If you have only one class implementing an interface, there is no gain, because it may always stay that way. There is no limit to the degree to which we can add interfaces and clutter the code.</p>

Title: Dependency Injection Confusion


问:
Mono is frequently used to say "Yes, .NET is cross-platform". How valid is that claim?
In What would you choose for your project between .NET and Java at this point in time? I say that I would consider the "Will you always deploy to Windows?" the single most important technical decision to make up front in a new web project, and if the answer is "no", I would recommend Java instead of .NET.

A very common counter-argument is that "If we ever want to run on Linux/OS X/Whatever, we'll just run Mono"1, which is a very compelling argument on the surface, but I don't agree for several reasons.

OpenJDK and all the vendor supplied JVM's have passed the official Sun TCK ensuring things work correctly.  I am not aware of Mono passing a Microsoft TCK.

Mono trails the .NET releases.  What .NET-level is currently fully supported?

Does all GUI elements (WinForms?) work correctly in Mono?

Businesses may not want to depend on Open Source frameworks as the official plan B.

I am aware that with the new governance of Java by Oracle, the future is unsafe, but e.g. IBM provides JDK's for many platforms, including Linux.  They are just not open sourced.

So, under which circumstances is Mono a valid business strategy for .NET-applications?

1Mark H summarized it as: "If the claim is that "I have a windows application written in .NET, it should run on mono", then not, it's not a valid claim - but Mono has made efforts to make porting such applications simpler."

答:
From a user perspective, Mono sucks at making Windows .NET apps compatible with Linux. If you actually try to run any decently written Windows app in Mono it wouldn't work.

From a packager's perspective (I used to do a bit of work at PortableApps), .NET can be both a blessing and a curse. If you're only on Windows, .NET makes deployment a lot easier because more libraries means less system-dependent stuff (most notably between versions of Windows, I believe) but is also extremely confusing even on Windows because .NET versions are neither backwards-compatible or forwards compatible. You have to download different versions of .NET for different programs.

That said, Mono is a fine starting point for making C# programs cross-platform. Just don't package the program with the latest WINE and call it done though. Look where that took Picasa.

As with the two languages/platforms political stability, RMS would probably start ranting on about how Mono is headed by Novell, whose honeymoon with Microsoft is quite notorious. Both platforms can be considered politcally unstable right now.

If you really want easy cross-platform, the tried and true path is QT which is quite stable politically as of right now it's a) LGPL'd, b) done with its major licensing issues years past, and c) backed by Nokia, which sells really really popular phones.

ContentLicense: CC BY-SA 2.5


问:
Minimizing Dependencies For GUIs
I've been working on a project, and have been charged with designing the projects GUI front-end. I'm coding in Java and using the Swing toolkit. Usability-wise, the GUI front-end follows all of Nielsen's heuristics. Users can easily get to where they want to go through the click of a button / JComboBox. Essentially, in Swing terms, what happens is their actions drive the creation/deletion of custom panels.

The GUI is coming along fine for the most part. However, I have to admit to being utterly dismayed at the tight web of dependencies my code is being smothered in.

The main problem that I've encountered, that I haven't been able to fix as of yet, is how to keep a reference to the panels/buttons being changed.

I'll give an example:

Say there's a button A

Say there's a panel B displaying picture C

Say there's another picture D (not currently being displayed by panel B)

When user clicks A, panel B should remove picture C and display picture D

My question is, what's the best way of keeping track of panel B? Since I need a global point of access to panel B, my solution has so far been to just shoehorn it into a static variable, and access it through a series of static getters and setters. And this static variable is usually stored in the reference's original class. I.e. UserPanel has a static variable that stores a reference to itself.

Is there an easy, tried-and-true way of dealing with these kinds of situations? Like my GUI works fine, but it is not modular and/or robust at all. To add to this, the dreaded 'cyclical dependencies' issue that's shunned by so many programmers is out here in full effect. I'm fairly new to development and just want to make sure that my code will be fairly extensible and won't cause much of a headache to the next person that decides to get a try at it.

I know there's loads of books out there that probably have a nice elegant solution to this, but unfortunately I just don't have the time to leisure read right now. I need something that's quick and dirty.

Thanks in advance

答:
have panel B add an ActionListener to Button A that (calls a method of B that) does the modifications

this way you don't need to keep access (this is done implicitly by the actionlistener)

then B should only know about the pictures it is displaying,

and the parent panel of B and A just needs to know that B is listening to A

Title: Minimizing Dependencies For GUIs


问:
Best alternative of Property file in Java
Hey I an working on the product which is live at multiple portals. The product is developed in GWT, JAVA, Hibernate.

My question is : Whether there is any alternative of using property file in java.

My Requirement :

For one property key there are multiple values in live portal for each different portal.

Each time I change property file, I need to make the war again.

The loading of any of the property should not be time-consuming.

Any help or suggession would be apprecialble !!!

答:
You may want to consider YAML

The Dropwizard framework in Java uses YAML for its external configuration. This is easier to work with than JSON since there are no braces.

You can use Jackson to unmarshal the YAML into DTOs with the Jackson Data format YAML module.

Here is an example YAML configuration for configuring the SLF4J within Dropwizard:

```<code>logging:

  level: WARN

  # Logger-specific levels.
  loggers:

    # Set specific levels
    "com.sun.jersey.api.client": DEBUG
    "com.yammer": INFO
    "org.multibit": DEBUG

  # ...
  # Settings for logging to stdout.
  console:

    # If true, write log statements to stdout.
    enabled: true

    # Do not display log statements below this threshold to stdout.
    threshold: ALL

  # Settings for logging to a file.
  file:

    # If true, write log statements to a file.
    enabled: false

    # Do not write log statements below this threshold to the file.
    threshold: ALL

    # The file to which current statements will be logged.
    currentLogFilename: /var/log/example/developer.log

    # When the log file rotates, the archived log will be renamed to this and gzipped. The
    # %d is replaced with the previous day (yyyy-MM-dd). Custom rolling windows can be created
    # by passing a SimpleDateFormat-compatible format as an argument: "%d{yyyy-MM-dd-hh}".
    archivedLogFilenamePattern: /var/log/example/developer-%d.log.gz

    # The number of archived files to keep.
    archivedFileCount: 5

    # The timezone used to format dates. HINT: USE THE DEFAULT, UTC.
    timeZone: UTC
</code>```

Body:
You may want to consider YAML

The Dropwizard framework in Java uses YAML for its external configuration. This is easier to work with than JSON since there are no braces.

You can use Jackson to unmarshal the YAML into DTOs with the Jackson Data format YAML module.

Here is an example YAML configuration for configuring the SLF4J within Dropwizard:

logging:

  level: WARN

  # Logger-specific levels.
  loggers:

    # Set specific levels
    "com.sun.jersey.api.client": DEBUG
    "com.yammer": INFO
    "org.multibit": DEBUG

  # ...
  # Settings for logging to stdout.
  console:

    # If true, write log statements to stdout.
    enabled: true

    # Do not display log statements below this threshold to stdout.
    threshold: ALL

  # Settings for logging to a file.
  file:

    # If true, write log statements to a file.
    enabled: false

    # Do not write log statements below this threshold to the file.
    threshold: ALL

    # The file to which current statements will be logged.
    currentLogFilename: /var/log/example/developer.log

    # When the log file rotates, the archived log will be renamed to this and gzipped. The
    # %d is replaced with the previous day (yyyy-MM-dd). Custom rolling windows can be created
    # by passing a SimpleDateFormat-compatible format as an argument: "%d{yyyy-MM-dd-hh}".
    archivedLogFilenamePattern: /var/log/example/developer-%d.log.gz

    # The number of archived files to keep.
    archivedFileCount: 5

    # The timezone used to format dates. HINT: USE THE DEFAULT, UTC.
    timeZone: UTC


问:
self referencing tables, good or bad?
Representing geographical locations within an application, the design of the underlying data model suggests two clear options (or maybe more?).

One table with a self referencing parent_id column
uk - london (london parent id = UK id)

or two tables, with a one to many relationship using a foreign key.

My preference is for one self-refercing table as it easily allows to extend into as many sub regions as required.

IN general do people veer away from self referencing tables, or are they A-OK ?

答:
It's a good idea if the relationship is actually hierarchical, and not a network relationship (for example a Bill of Materials is a network relationship, not a hierarchical one).

It can be slow to query. To speed things up, you can use a Closure Table.

http://karwin.blogspot.ca/2010/03/rendering-trees-with-closure-tables.html

Title: self referencing tables, good or bad?


问:
Does anybody ever uses the OpenId url Log in?
As programmers I think we are the most inclined to use new technologies and things that are not so mainstream.

I absolutely love OpenId's ability to log in with credentials you already have, but I always use a provider that I already use and that is the fastest to log in, like google. God knows I'd never used it if i had to type in the whole "https://www.google.com/accounts/o8/id" I like to click on the google button and then just say yes (the first time, then its just clicking the button).

Does any of you ever uses the Generic "Enter your own provider" OpenId?

Do you think its worth to even offer it to more casual end users?

答:
When I signed up to Stack Overflow they didn't have the big buttons they do now (I likely would've used Google). So I signed up for MyOpenID and still use it. It's simple to remember the URL, but it's even easier now that SO has a big "MyOpenID" button alongside Google etc.

Even if you're covering the majority of bases (as per your comment) it's not like it's any trouble to add an "enter your own" box - I mean, the Google/Yahoo boxes just link to the appropriate OpenID URLs anyway.


问:
What are some good assignments for an introductory course, to reach today's students?
I periodically teach an introduction to programming course using Java. I want to give my students some exciting assignments that they can relate to or find interesting. At the very least, I want assignments that make sense and have an internal cohesion (for example, some very bad programming exercises seem contrived just so that you have to use the most recent programming construct covered).

To give you an idea of scope, here's what's being covered:

The assignments must be in Java, using some external library can be done, but it would need to be a simple API and not a full framework

Variables, Primitives and Strings

Console input and output

if, for, while

Arithmetic and logical operators

Simple graphics with line and shape drawing

Static methods

One-dimensional arrays

The students will not go into advanced topics (e.g., no recursion, no emphasis on inheritance). Thus, I'm not looking for complex projects: "Have them write a C compiler. I did it when I was 5 and it was the only way I learned!"

If you have ideas that are more advanced than the scope, please post them on the "Challenging" question linked below, and not this one.

Instead, what I'm looking for are interesting ideas that can be accomplished in a short program. For example:

Students can write a console version of the "Star Wars Name" Generator. This is effectively reading Strings and using substring, but I think it's fun. A similar variation would be a "Mad Libs" program. I used this one five years ago, and perhaps it's already "stale."

Using some trig supplied to them, students can draw regular polygons, and create interesting spiral shapes.

It's also not out of the question to use some simple animation class with most of the code supplied to them. And if you know a Twitter or Facebook service that can be accessed with a simple API, I would be very interested to know.

Please note that this question is different from the "Challenging Java questions for beginners" Question. I'm not looking for challenging per se, just interesting. I think if students work on something where they can easily answer "why would anyone ever want to program that?" then they will learn better.

Even simple problems like computing Miles per Gallon can be good. Although, if there's a way to update it to be more relevant, all the better. If you have an exercise from somewhere else (a book or a website), please cite the source.

To help you think about the typical freshman today, check out the Beloit Mindset list, for many examples that will surely make you feel old.

答:
Given the constraints, I'd suggest implementing a version of Hangman. It would allow for the students to demonstrate all of the techniques you are introducing them to, without being overly complex.

It can also be used as a developing project as the course progresses.  e.g.  once you have covered strings and variables starts out as a text version

e.g.

```<code>You have 10 guesses left.      * * * E * * T
What is your next guess?
</code>```

then introduce loops to remove the cut and paste element from the code as the 10 guesses count down...  building up to having line graphics and the stick person being hung / saved at the end of the 5 week course.

Like most other people who have experience of recruiting and interviewing programmers, it really makes me cringe that this level of tuition is needed at university, but alas, it probably will continue to be needed until schools treat programming as a serious subject on a par with mathematics or sciences

Body:
Given the constraints, I'd suggest implementing a version of Hangman. It would allow for the students to demonstrate all of the techniques you are introducing them to, without being overly complex.

It can also be used as a developing project as the course progresses.  e.g.  once you have covered strings and variables starts out as a text version

e.g.

You have 10 guesses left.      * * * E * * T
What is your next guess?

then introduce loops to remove the cut and paste element from the code as the 10 guesses count down...  building up to having line graphics and the stick person being hung / saved at the end of the 5 week course.

Like most other people who have experience of recruiting and interviewing programmers, it really makes me cringe that this level of tuition is needed at university, but alas, it probably will continue to be needed until schools treat programming as a serious subject on a par with mathematics or sciences


答:
IF YOU ARE LOOKING FOR A GLOBALLY ACCEPTED FORMAT

MISRA/JSF/AUTOSAR covers nearly 100% of any and every industry-standard for naming and organising C/C++ code. The problem is that they will not be for free to get hold of i.e. each of the guidebooks costs some money. I know that MISRA 2008 C/C++ coding standard book probably costs about 50 USD.

You can think of these as the Harvard Referencing for bibliography and addition reading when you write a journal. I have used MISRA and it is a good way to name your functions and variables, and organise them for proper use.

IF YOU ARE LOOKING FOR SOMETHING TEMPORARY

The references you provided for Python and Java are okay I guess. I have seen people adopting javadoc style commenting, naming, and organising of code. As a matter of fact, in my last project, I had to write C++ code in Java-like functions/variable names. Two reasons behind this:

1) It was apparently easier to follow.

2) Production code requirements did not touch the ground of safety-critical software system standards.

3) Legacy code was (somehow) in that format.

4) Doxygen allowed Javadoc sytle commenting. At that moment, we were using doxygen to generate documentation for the production guys.

Many programmers will be opponent of this, but I personally thing that there is nothing wrong with adopting javadoc style function/variable naming in C/C++. YES OF COURSE, the practices of organising your flow control, thread safety, etc. needs to be addressed regardless. However, I am not an applicant here. I also don't know how stringent your production code format requirements are. Without diverting it to an off-topic area, I suggest you review your requirements, find out how depended you are on a specific naming convention, and go with a solution mentioned in mine and others' answers

Hope this helped!?

Title: Naming conventions used for variables and functions in C

Body:
While coding a large project in C I came upon a problem. If I keep on writing more code then there will be a time when it will be difficult for me to organize the code. I mean that the naming for functions and variables for different parts of the program may seem to be mixed up.

So I was thinking whether there are useful naming conventions that I can use for C variables and functions?

Most languages suggest a naming convention. But for C the only thing I have read so far is the names should be descriptive for code readability.

EDIT:

Examples of some examples of suggested naming conventions:

Python's PEP 8

Java Tutorial

I read some more naming conventions for java somewhere but couldn't remember where.


答:
Passing in an implementation of the interface should be preferred. The real benefits come with more complex examples. What happens if the creation is determined by runtime behavior? Passing in a factory rather than the instance solves that, but can get unweildy. And what if the creation timeframe was not clear cut (the consuming class could not just call the factory, but would have to wait until you set everything up).

Further, what happens when the class you're instantiating isn't the concrete version either? If all you need is a IFoo some factory is providing the implementation. Some concrete implementations need some dependencies injected and others need others (or none).

IoC containers are a giant hammer. Usually just passing in the concrete implementation of an interface is clear, functional, and flexible. But sometimes the design grows complex enough that you need IoC containers to manage the tangle of dependencies spread throughout your object graph.

Title: Dependency Injection Confusion

Body:
I think I have a decent grasp of what Dependency Inversion principle (DIP) is, my confusion is more around dependency injection.

My understanding is the whole point of DI is to decouple parts of an application, to allow changes in one part without effecting another, assuming the interface does not change.

For examples sake, we have this

```<code>public class MyClass(IMyInterface interface)
{
   public MyClass
   {
      interface.DoSomething();
   }
}

public interface IMyInterface
{
    void DoSomething();
}
</code>```

How is this

```<code>var iocContainer = new UnityContainer();
iocContainer.Resolve<MyClass>();
</code>```

better practice than doing this

```<code>//if multiple implementations are possible, could use a factory here.

IMyInterface interface = new InterfaceImplementation();
var myClass  = new MyClass(interface);
</code>```

It may be I am missing a very important point, but I am failing to see what is gained. I am aware that using an IOC container I can easily handle an objects life cycle, which is a +1 but I don't think that is core to what IOC is about.

Edit

Here is an expanded example:

```<code>void Main()
{
    IRepository repository = new AddRepository();
    var maths = new PointlessMaths(repository,5);
    Console.WriteLine(maths.Result);
}

public interface IRepository
{
    int DoSomething(int id);
}

public class AddRepository : IRepository
{
    public int DoSomething(int id)
    {
        return id + 1;
    }
}

public class SubtractRepository : IRepository
{
    public int DoSomething(int id)
    {
        return id - 1;
    }
}

public class PointlessMaths
{
    public int Result {get;set;}
    public PointlessMaths(IRepository repository, int id)
    {
        Result = repository.DoSomething(id);
    }
}
</code>```


问:
Distinction between API and frontend-backend
I'm trying to write a "standard" business website. By "standard", I mean this site runs the usual HTML5, CSS and JavaScript for the front-end, a back-end (to process stuff), and runs MySQL for the database. It's a basic CRUD site: the front-end just makes pretty whatever the database has in store; the backend writes to the database whatever the user enters and does some processing. Just like most sites out there.

In creating my GitHub repositories to begin coding, I've realized I don't understand the distinction between the front-end back-end, and the API. Another way of phrasing my question is: where does the API come into this picture?

I'm going to list some more details and then questions I have - hopefully this gives you guys a better idea of what my actual question is because I'm so confused that I don't know the specific question to ask.

Some more details:

I'd like to try the Model-View-Controller pattern. I don't know if this changes the question/answer.

The API will be RESTful

I'd like my back-end to use my own API instead of allowing the back-end to cheat and call special queries. I think this style is more consistent.

My questions:

Does the front-end call the back-end which calls the API? Or does the front-end just call the API instead of calling the back-end?

Does the back-end just execute an API and the API returns control to the back-end (where the back-end acts as the ultimate controller, delegating tasks)?

Long and detailed answers explaining the role of the API alongside the front-end back-end are encouraged. If the answer depends on the model of programming (models other than the Model-View-Controller pattern), please describe these other ways of thinking of the API.

答:
Let us sketch out a "typical" website's architecture, with both a "front-end" and a "back-end". And since it's a website, we'll also explicitly have a "client". (Since there's no way for JavaScript in a browser to call MySQL on a server directly.)

For clarity, the terms we're using are:

Client: The HTML5 compliant browser, esp. the DOM and the JavaScript there loaded to manipulate it.

Front-End: The PHP server that the DOM is pointed to, containing both the individual page requested and some AJAX style XML or JSON access points.

Back-end: A database server, where MySQL runs.

For a properly designed program, each of these components has a private API to communicate with the others. The "front-end" PHP code does not issue arbitrary SQL SELECT statements directly, but rather calls stored procedures, pre-authorized SQL, or even distinct PHP calls to an entirely different instance of PHP that runs on the back-end server. These stored procedures or distinct HTTP calls are themselves an API.

The definition doesn't change even if we allow for some impurity of our design. If your PHP file writes and sends a SQL string directly to MySQL, IT IS STILL AN API, albeit a very unusual one that you are unlikely to repeat.

Note that it's entirely possible to have your front-end PHP be strictly synchronous, with no AJAX voodoo whatsoever.  If you call the same external PHP functions in said synchronous file, you could consider them as using the same API as the client-side version, although use of the term "API" here may not give any real clarity.

An API as an Application Programming Interface, after all, and really refers to any time one program calls outside of its own process.  If you're writing a project that has both a front-end and a back-end, be those AJAX/PHP/MySQL as above or MS Access/SQL Server, it's worthwhile to specify explicity how you'll be calling each other, if for no other reason than to make it easy to know where to look when something breaks.

(And the topic of public API is something else entirely.  In our example above, only the URL displayed in the client is a "public API."  Everything else is, in essence, "private."  As in, you don't expect any code beyond your control to call your internal API, and you either deny such results outright or reserve the right to do so in the future.


答:
Point by point:

OpenJDK and all the vendor supplied JVM's have passed the official Sun TCK ensuring things work correctly. I am not aware of Mono passing a Microsoft TCK.

There is a specification that the Microsoft CLR conforms to.  Mono is not a clone of Microsoft's CLR, but rather an implementation of that same specification.  On the one hand, there's the chance that this results in even greater incompatibility.  In practice, this has worked very well for mono.

Mono trails the .NET releases. What .NET-level is currently fully supported?

Because the documentation for .Net precedes the actual release, mono actually had completed (not a beta release) support for .Net 4 out before the official Microsoft release.  Additionally, mono does a very good job of documenting what things are not supported, and they have a few tools you can run against existing project to help you spot places with the potential for incompatibility.

Does all GUI elements (WinForms?) work correctly in Mono?

The vast majority of winforms works just fine.  WPF, not so much.  I've heard that the same is true for Java — many of the advanced widget/gui toolkits aren't as well supported on all platforms.

Businesses may not want to depend on Open Source frameworks as the official plan B.

If that's the case, they definitely won't be going with Java, then, as that puts an open source framework up even higher at plan A.

In summary, if you want to build a cross-platform app right now, there's nothing wrong with starting with mono from the get-go and using that as your framework.  You can even deploy mono on Windows if you want.

On the other hand, if you you're looking at building a Windows app today that you think might need to be cross platform at some unknown point in the future, you probably want to go with the native Windows widgets and tools.  .Net does a much better job here than does Java, and mono is a perfectly acceptable fall back in this scenario.

In other words: Yes, .Net if cross-platform in every way that matters to your question.  No, mono won't just run every .Net program out of the box, but your question is in the context of a new project. It doesn't take much work to keep new projects out of trouble and avoid compatibility issues, especially if you think in terms of developing for mono rather than developing for .Net.

Most of all, though, I think this is the wrong question in the first place.  Unless you're building a new development team from scratch, you're starting with programmers that have expertise in one area or another.  Your best results will be achieved by going with what your programmers already know.  As important as building a cross-platform app may seem, if you have a team full of .Net programmers with little java experience, you're unlikely to fire them or make them all learn java for your next project.  If you have a team full of java programmers, you're not gonna ask them to learn .Net for a Windows-only project.  Either of those would be nuts.

Title: Mono is frequently used to say "Yes, .NET is cross-platform". How valid is that claim?

Body:
In What would you choose for your project between .NET and Java at this point in time? I say that I would consider the "Will you always deploy to Windows?" the single most important technical decision to make up front in a new web project, and if the answer is "no", I would recommend Java instead of .NET.

A very common counter-argument is that "If we ever want to run on Linux/OS X/Whatever, we'll just run Mono"1, which is a very compelling argument on the surface, but I don't agree for several reasons.

OpenJDK and all the vendor supplied JVM's have passed the official Sun TCK ensuring things work correctly.  I am not aware of Mono passing a Microsoft TCK.

Mono trails the .NET releases.  What .NET-level is currently fully supported?

Does all GUI elements (WinForms?) work correctly in Mono?

Businesses may not want to depend on Open Source frameworks as the official plan B.

I am aware that with the new governance of Java by Oracle, the future is unsafe, but e.g. IBM provides JDK's for many platforms, including Linux.  They are just not open sourced.

So, under which circumstances is Mono a valid business strategy for .NET-applications?

1Mark H summarized it as: "If the claim is that "I have a windows application written in .NET, it should run on mono", then not, it's not a valid claim - but Mono has made efforts to make porting such applications simpler."


答:
I think you're being confused by the way the term API is being misused and abused by many web developers.

API means Application Programming Interface, i.e. any officially specified interface between different systems (or parts of the same system).

Some time ago, it became a big thing for web startup to offer public access to some of their internal data through a web service API, typically using REST and JSON, thus allowing third-party developers to integrate with their systems. Web developers started using the term "API" to mean specifically (and only) "publically accessible web service", and misusing it to include the implementation thereof.

In terms of frontend and backend, this web service API (and its implementation) is the backend. Some parts of it may be publically accessible and others only to your frontend.

A different name for this is "service layer", i.e. code that

represents services which the frontend calls

contains no display logic (that's the job of the frontend, after all)

is more abstract and coarse grained than simple CRUD actions (one service call will often involve multiple CRUD actions and should be executed within a database transaction).

contains the business logic of the application

Title: Distinction between API and frontend-backend

Body:
I'm trying to write a "standard" business website. By "standard", I mean this site runs the usual HTML5, CSS and JavaScript for the front-end, a back-end (to process stuff), and runs MySQL for the database. It's a basic CRUD site: the front-end just makes pretty whatever the database has in store; the backend writes to the database whatever the user enters and does some processing. Just like most sites out there.

In creating my GitHub repositories to begin coding, I've realized I don't understand the distinction between the front-end back-end, and the API. Another way of phrasing my question is: where does the API come into this picture?

I'm going to list some more details and then questions I have - hopefully this gives you guys a better idea of what my actual question is because I'm so confused that I don't know the specific question to ask.

Some more details:

I'd like to try the Model-View-Controller pattern. I don't know if this changes the question/answer.

The API will be RESTful

I'd like my back-end to use my own API instead of allowing the back-end to cheat and call special queries. I think this style is more consistent.

My questions:

Does the front-end call the back-end which calls the API? Or does the front-end just call the API instead of calling the back-end?

Does the back-end just execute an API and the API returns control to the back-end (where the back-end acts as the ultimate controller, delegating tasks)?

Long and detailed answers explaining the role of the API alongside the front-end back-end are encouraged. If the answer depends on the model of programming (models other than the Model-View-Controller pattern), please describe these other ways of thinking of the API.


问:
Full Text Search Strategy For My Website
I have a website that allows users to search for items in various categories. Each category is a separate area (page) of my website. For example, some categories might be cars, bikes, books etc. At the moment a user has to search for an item by going to the page (for example, cars) and searching for the car they want. I would like to allow the user to search for anything on my site, from my main home page.

At the moment, each page (category) has its own set of tables, and I don't really want to turn Full Text Search on for each table (20+ of them) and search each table individually when a search is done. This is going to be slow and tedious.

What I'm thinking of doing is creating a single table that will hold all searchable information for each category of item (when an item is saved in its respective table, I would copy all searchable information over to my 'Search' table). I would then turn Full Text Search on for that table, and search that table.

Does this sound reasonable? Is there a better way? I've never used Full Text Search before, so this is new to me.

答:
Well, for starters it's worth wondering why you have distinct tables for each category, instead of sharing your tables between categories.  Were I to re-design your database from scratch, I'd have a three-aspect table structure:

Categories (car/bike/etc.)

Items (i.e., a thing that belongs to a category)

Item Properties (a thing about an item that's searchable.)

You'd have indexes on Item-Properties, for both the property name and its value, and a full-text search on the item's text.

Additional tables could be added for Property-Type and "full text", or those could be built into the tables above.  (That's an optimization question, after all, and premature optimization is the root of all evil.)


问:
What would be the fastest way of storing or calculating legal move sets for chess pieces?
For example if a move is attempted I could just loop through a list of legal moves and compare the x,y but I have to write logic to calculate those at least every time the piece is moved.

Or, I can store in an array [,] and then check if x and y are not 0 then it is a legal move and then I save the data like this
[0][1][0][0] etc etc for each row where 1 is a legal move, but I still have to populate it.

I wonder what the fastest way to store and read a legal move on a piece object for calculation.

I could use matrix math as well I suppose but I don't know.

Basically I want to persist the rules for a given piece so I can assign a piece object a little template of all it's possible moves considering it is starting from it's current location, which should be just one table of data. But is it faster to loop or write LINQ queries or store arrays and matrices?

```<code>public class Move
{
    public int x;
    public int y;
}

public class ChessPiece : Move
{
    private List<Move> possibleMoves { get; set; }

    public bool LegalMove(int x, int y){
      foreach(var p in possibleMoves)
      {
         if(p.x == x && p.y == y)
         {  return true;  }
      }
    }
}
</code>```

Anyone know?

答:
the easiest way would be to have the LegalMove(Move) use custom code for each different piece

that way you just need to check that abs(x-m.x)==abs(y-m.y) for a bishop for example (not counting blocking pieces)

or if you have a reference to the board and position in the piece then some custom logic would be (again for bishop):

```<code>IEnumerable LegalMoves(){
    for(int xl = x+1,yl = y+1; yl<8 && xl<8; xl++,yl++){
        if(board[xl][yl].isFree())
            yield return Move(xl,yl);
        else if(board[xl][yl].HasEnemy()){
            yield return Move(xl,yl);
            break;//can't jump over enemy
        }else break;//can't jump over own pieces
    }
    //mirror it for each of the 4 sides
}
</code>```

Body:
the easiest way would be to have the LegalMove(Move) use custom code for each different piece

that way you just need to check that abs(x-m.x)==abs(y-m.y) for a bishop for example (not counting blocking pieces)

or if you have a reference to the board and position in the piece then some custom logic would be (again for bishop):

IEnumerable LegalMoves(){
    for(int xl = x+1,yl = y+1; yl<8 && xl<8; xl++,yl++){
        if(board[xl][yl].isFree())
            yield return Move(xl,yl);
        else if(board[xl][yl].HasEnemy()){
            yield return Move(xl,yl);
            break;//can't jump over enemy
        }else break;//can't jump over own pieces
    }
    //mirror it for each of the 4 sides
}


答:
The fastest way, I think, would be to keep an array of legal moves for every piece. When you move a piece, you update the legal moves of the other pieces. For the square that you left, you add new moves to the pieces that can now move there. For the new square, you remove legal moves from the pieces that could move there before.

So (in pseudo code):

```<code>move piece from X0,Y0 to X1,Y1
with X0,Y0:
    check surrounding squares for king/pawn moves, add legal move where appropriate
    check the knight positions, add legal move where appropriate
    check horizontal/vertical lines for rook/queen, add legal move where appropriate
    check diagonal lines for queen/bishops, add legal move where appropriate
with X1,Y1:
    same checks, but now remove legal move
</code>```

Of course, checks along lines should stop when they encounter another piece that is blocking the path.

It might be handy to keep some flag on the squares, so that you know if special reachability rules apply to the pawns or king/rook.

Somewhere along the line, when pieces get removed, this will get slower than just checking legal moves for the pieces themselves. At that point you might switch to a straight-forward check for the moving piece itself, abandoning updating the other pieces.

As for how to store it, a 64-bit mask would be enough to encode an 8x8 board.

```<code>piece.removeLegalMove(square_mask):
    mymask &= ~square_mask

pieace.addLegalMove(square_mask):
    mymask |= square_mask;

piece.canMove(x, y):
    square_mask = 1 << (x + 8 * y);
    return mymask & square_mask > 0;
</code>```

But I'm fairly unfamiliar with C#, so I don't know if it handles bit masks. I assume it does.

Title: What would be the fastest way of storing or calculating legal move sets for chess pieces?

Body:
For example if a move is attempted I could just loop through a list of legal moves and compare the x,y but I have to write logic to calculate those at least every time the piece is moved.

Or, I can store in an array [,] and then check if x and y are not 0 then it is a legal move and then I save the data like this
[0][1][0][0] etc etc for each row where 1 is a legal move, but I still have to populate it.

I wonder what the fastest way to store and read a legal move on a piece object for calculation.

I could use matrix math as well I suppose but I don't know.

Basically I want to persist the rules for a given piece so I can assign a piece object a little template of all it's possible moves considering it is starting from it's current location, which should be just one table of data. But is it faster to loop or write LINQ queries or store arrays and matrices?

```<code>public class Move
{
    public int x;
    public int y;
}

public class ChessPiece : Move
{
    private List<Move> possibleMoves { get; set; }

    public bool LegalMove(int x, int y){
      foreach(var p in possibleMoves)
      {
         if(p.x == x && p.y == y)
         {  return true;  }
      }
    }
}
</code>```

Anyone know?

Body:
The fastest way, I think, would be to keep an array of legal moves for every piece. When you move a piece, you update the legal moves of the other pieces. For the square that you left, you add new moves to the pieces that can now move there. For the new square, you remove legal moves from the pieces that could move there before.

So (in pseudo code):

move piece from X0,Y0 to X1,Y1
with X0,Y0:
    check surrounding squares for king/pawn moves, add legal move where appropriate
    check the knight positions, add legal move where appropriate
    check horizontal/vertical lines for rook/queen, add legal move where appropriate
    check diagonal lines for queen/bishops, add legal move where appropriate
with X1,Y1:
    same checks, but now remove legal move

Of course, checks along lines should stop when they encounter another piece that is blocking the path.

It might be handy to keep some flag on the squares, so that you know if special reachability rules apply to the pawns or king/rook.

Somewhere along the line, when pieces get removed, this will get slower than just checking legal moves for the pieces themselves. At that point you might switch to a straight-forward check for the moving piece itself, abandoning updating the other pieces.

As for how to store it, a 64-bit mask would be enough to encode an 8x8 board.

piece.removeLegalMove(square_mask):
    mymask &= ~square_mask

pieace.addLegalMove(square_mask):
    mymask |= square_mask;

piece.canMove(x, y):
    square_mask = 1 << (x + 8 * y);
    return mymask & square_mask > 0;

But I'm fairly unfamiliar with C#, so I don't know if it handles bit masks. I assume it does.


答:
Depending if you have the freedom to do so, I would suggest using something designed for searching rather than doing full-text searches in database. Databases aren't really designed for full-text searches, so the performance isn't going to be the best and can put a lot of additional load onto it.

I would suggest taking a look at Solr, ElasticSearch, or Sphinx. They will allow you to do far more customized searches than a full text database search. They all offer ways of importing data from a database and can do indexing based on the fields you provide (important if you want a brand to be indexed differently than a product name, for instance).

Title: Full Text Search Strategy For My Website

Body:
I have a website that allows users to search for items in various categories. Each category is a separate area (page) of my website. For example, some categories might be cars, bikes, books etc. At the moment a user has to search for an item by going to the page (for example, cars) and searching for the car they want. I would like to allow the user to search for anything on my site, from my main home page.

At the moment, each page (category) has its own set of tables, and I don't really want to turn Full Text Search on for each table (20+ of them) and search each table individually when a search is done. This is going to be slow and tedious.

What I'm thinking of doing is creating a single table that will hold all searchable information for each category of item (when an item is saved in its respective table, I would copy all searchable information over to my 'Search' table). I would then turn Full Text Search on for that table, and search that table.

Does this sound reasonable? Is there a better way? I've never used Full Text Search before, so this is new to me.