问:
Do you have your own 'misc utils' library? What part are you most proud of?
I know that many of us maintain our own little personal library with tools and utilities that we use often.

I've had mine since I was 16 years old so it has grown to quite a considerable size. Some of the stuff I've written has since been added to the framework. I wrote my own little implementation of expression trees for use with genetic algorithms long before LINQ, which I quite like and was proud of at the time - of course its pretty useless now. But recently I have been going through it and upgrading to .NET 4.0 and re-kindled an interest.

So I'm curious as to what you use your library for. Maybe we could get some cool ideas going for useful little snippets and share them amongst ourselves.

So my questions are:

Do you have a miscellaneous utility library?

Which part are you most proud of and why?

Give an example of code if you like :-)

答:
K Combinator (C#, Scala)

I use the K combinator in Ruby quite often, mostly in folds when the folding operation is performed through a side effect rather than a return value, like in this example:

```<code>some_collection.reduce(Hash.new(0)) {|acc, el| acc[el] += 1 }
</code>```

This counts how often each element occurs in some_collection. Unfortunately, it doesn't actually work, since the block has to return the new value of the accumulator at each iteration, but in Ruby assignments evaluate to the assigned value.

So, you have to ecplicitly return the new value of the accumulator like this:

```<code>some_collection.reduce(Hash.new(0)) {|acc, el| acc[el] += 1; acc }
</code>```

But I find such explicit sequencing ugly in this functional-ish style using folds. The K combinator (called Object#tap in Ruby) to the rescue:

```<code>some_collection.reduce(Hash.new(0)) {|acc, el| acc.tap { acc[el] += 1 }}
</code>```

I have already missed it a couple of times in C# (mostly because for some reason collection mutators such as List.Add return void instead of this) and Scala, so I carry around this:

```<code>namespace GenericExtensions
{
    public static class GenericExtensions
    {
        public static T Tap<T>(this T o, Action<T> f)
        {
            Contract.Requires(o != null);
            Contract.Requires(f != null);

            f(o);
            return o;
        }

        public static T Tap<T>(this T o, Action f)
        {
            Contract.Requires(o != null);
            Contract.Requires(f != null);

            f();
            return o;
        }
    }
}
</code>```

and in Scala:

```<code>class Tap[T](o: T) {
  def tap(f: T => Unit) = { f(o); o }
  def tap(f: => Unit) = { f; o }
}

object Implicits { implicit def any2Tap[T](o: T) = new Tap(o) }
</code>```

Identity Function (Ruby)

Something I am missing in Ruby, is a nicely named way to access the identity function. Haskell provides the identity function under the name of id, Scala under the name of identity. This allows one to write code like:

```<code>someCollection.groupBy(identity)
</code>```

The equivalent in Ruby is

```<code>some_collection.group_by {|x| x }
</code>```

Doesn't exactly roll off the tongue, does it?

The fix is

```<code>IDENTITY = -> x { x }

some_collection.group_by(&IDENTITY)
</code>```

ForEach (.NET)

Another sorely missing method in C#:

```<code>namespace IEnumerableExtensions
{
    public static class IEnumerableExtensions
    {
        public static void ForEach<T>(this IEnumerable<T> xs, Action<T> f)
        {
            Contract.Requires(xs != null);
            Contract.Requires(f != null);

           foreach (var x in xs) f(x);
        }
    }
}
</code>```

Body:
K Combinator (C#, Scala)

I use the K combinator in Ruby quite often, mostly in folds when the folding operation is performed through a side effect rather than a return value, like in this example:

some_collection.reduce(Hash.new(0)) {|acc, el| acc[el] += 1 }

This counts how often each element occurs in some_collection. Unfortunately, it doesn't actually work, since the block has to return the new value of the accumulator at each iteration, but in Ruby assignments evaluate to the assigned value.

So, you have to ecplicitly return the new value of the accumulator like this:

some_collection.reduce(Hash.new(0)) {|acc, el| acc[el] += 1; acc }

But I find such explicit sequencing ugly in this functional-ish style using folds. The K combinator (called Object#tap in Ruby) to the rescue:

some_collection.reduce(Hash.new(0)) {|acc, el| acc.tap { acc[el] += 1 }}

I have already missed it a couple of times in C# (mostly because for some reason collection mutators such as List.Add return void instead of this) and Scala, so I carry around this:

namespace GenericExtensions
{
    public static class GenericExtensions
    {
        public static T Tap<T>(this T o, Action<T> f)
        {
            Contract.Requires(o != null);
            Contract.Requires(f != null);

            f(o);
            return o;
        }

        public static T Tap<T>(this T o, Action f)
        {
            Contract.Requires(o != null);
            Contract.Requires(f != null);

            f();
            return o;
        }
    }
}

and in Scala:

class Tap[T](o: T) {
  def tap(f: T => Unit) = { f(o); o }
  def tap(f: => Unit) = { f; o }
}

object Implicits { implicit def any2Tap[T](o: T) = new Tap(o) }

Identity Function (Ruby)

Something I am missing in Ruby, is a nicely named way to access the identity function. Haskell provides the identity function under the name of id, Scala under the name of identity. This allows one to write code like:

someCollection.groupBy(identity)

The equivalent in Ruby is

some_collection.group_by {|x| x }

Doesn't exactly roll off the tongue, does it?

The fix is

IDENTITY = -> x { x }

some_collection.group_by(&IDENTITY)

ForEach (.NET)

Another sorely missing method in C#:

namespace IEnumerableExtensions
{
    public static class IEnumerableExtensions
    {
        public static void ForEach<T>(this IEnumerable<T> xs, Action<T> f)
        {
            Contract.Requires(xs != null);
            Contract.Requires(f != null);

           foreach (var x in xs) f(x);
        }
    }
}


问:
Javascript Rookie Question: Define Variables Inline
I'm proficient with HTML and CSS but I'm still pretty shaky when it comes to Javascript. That said, I've been able to build a site using the Internet Archive Book Reader, which relies on reader.js

Here's a copy of one of my versions of reader.js https://gist.github.com/dylan-k/ed4efed2384e221d46cc

It's a good site, but I find I have to repeat things a lot. Basically, I have one copy of reader.js for every page/book featured on the site. It seems there must be a better way. I re-use the script, making copies, just so that I can change lines 28, 80, 83, 84.

Is there a way I could include just one copy of reader.js and then use a

答:
As you've noticed, the basic problem is that - as written - that script isn't reusable. It's hardcoded for a specific case. There's no straightforward way to pass in different settings on different pages.

So you'll need to modify that script to make it reusable.

There are any number of ways you could do that, but here's one that involves minimal modifications to the original code:

Then, change the following lines in your modified version of reader.js:

Line 28 becomes:

var url = BookReaderParams.urlStart + leafStr.replace(re, imgStr) + BookReaderParams.urlEnd;

Line 80 becomes:

br.numLeafs = BookReaderParams.numLeafs;

And lines 83 and 84 become:

br.bookTitle = BookReaderParams.bookTitle;
br.bookUrl = BookReaderParams.bookUrl;

This isn't how I would structure things if I were writing this functionality from the ground up; there are much cleaner patterns for reusable code. However, given the codebase you're working with, this is a quick-and-dirty way to inject values into the script without needing a separate copy of reader.js for every page.

ContentLicense: CC BY-SA 3.0

Body:
As you've noticed, the basic problem is that - as written - that script isn't reusable. It's hardcoded for a specific case. There's no straightforward way to pass in different settings on different pages.

So you'll need to modify that script to make it reusable.

There are any number of ways you could do that, but here's one that involves minimal modifications to the original code:

<script>

// Define a global object containing the BookReader parameters for this
// page. Make sure this appears in your markup *before* the <script> tag
// that loads reader.js.
var BookReaderParams = {
    urlStart: "/exhibitions/bythebook/book8/W922/W922_",
    urlEnd: "_sap.jpg",
    numLeafs: 328,
    bookTitle: "Liber Amicorum (Friendship Book) of Joannes Carolus Erlenwein",
    bookUrl  = "http://thewalters.org/exhibitions/bythebook/book8/"
};

</script>
<script src="reader-modified.js"></script>

Then, change the following lines in your modified version of reader.js:

Line 28 becomes:

var url = BookReaderParams.urlStart + leafStr.replace(re, imgStr) + BookReaderParams.urlEnd;

Line 80 becomes:

br.numLeafs = BookReaderParams.numLeafs;

And lines 83 and 84 become:

br.bookTitle = BookReaderParams.bookTitle;
br.bookUrl = BookReaderParams.bookUrl;

This isn't how I would structure things if I were writing this functionality from the ground up; there are much cleaner patterns for reusable code. However, given the codebase you're working with, this is a quick-and-dirty way to inject values into the script without needing a separate copy of reader.js for every page.


问:
Code base migration - old versioning system to modern
Our current code base is contained in a versioning system that is old and outdated (Visual Sourcesafe 5.0, mid 1990's), and contains a mix of packages that are no longer used, ones that are being used but no longer updated, and newer code. It is also a mix of 4 languages, and includes libraries for some of our systems (Such as Dialogic, Sun Tzu {clipper}) implementations. This breaks down into the following categories:

Legacy code - No longer used (Systems that have been retired or replaced, etc)

Legacy code - In current use (No intentions for upgrades or minor bug fixes, only major fixes if needed)

Current code - In current use, and will be used for future versions/development

Support libraries - For both legacy and current code (Some of the legacy libraries are no longer available as well)

We would like to migrate this to a newer versioning system as we will be adding more developers, and expanding the reach to include remote programmers.

When migrating, how do you structure it? Do you just perform a dump of all the data and then import it into the new system, or do you segregate according to type before you bring it into the new system? Do you set up a separate area for libraries, or keep them with the relevant packages? Do you separate by language, system, both? A general outline and methodology is fine, it doesn't need to be broken down to individual program level.

答:
It sounds like you've got some flexibility in how you migrate your codebase.

One option would be to make a "clean break" where you set up the new system according to the newer best practices that you want to adhere to. In that case, you can just export everything you want from the old system and import into the new based on whatever structure makes sense. You don't have to tie yourself down to old (90's?) decisions.

The other extreme is to try to migrate to the new system while being strict about keeping as much legacy structure as possible. For example, you might want to import all the Visual Sourcesafe history into the new system (commit messages, authors, etc.) Then you slowly refactor your way into whatever newer structures/organization that you want.

There are pros and cons to both extremes and you can always choose something in the middle. You'll want to consider what kinds of requirements you have (or want) in place. For example:

Do you need the full commit history or can you drop it (look at Sourcesafe if necessary)?

What level of consistency are your developers (or others accessing the repository) expecting? Will restructuring be a problem?

What kinds of differences exist between Sourcesafe and the new version control system? E.g. centralized vs. distributed. How will that affect your decisions?

So as for a general outline and methodology, I'm not sure one really exists for so many unknown variables (such as what system you're moving to, or additional details about your project). If you provide more detail then you might get a better answer.

But at the end of the day it's really up to you and your team to decide what's best.

Just off-hand, based on what detail you provided, I'd personally lean toward the "clean break" extreme and use a DVCS like git or mercurial. Use multiple repositories for each logical grouping/libary/system. Spend a little time to get it all set up in a new, fresh way that will allow your new developers to get started without the baggage of the old structure.


问:
Why is a software development life-cycle so inefficient?
Currently the software development lifecycle followed in the IT company I work at is:

The "Business" works with a solution manager to build a Business Requirement document

The solution manager works with the Program manager to build a Functional Spec

The PM works with the engineering lead to develop a release plan and with the engineering team to develop technical specifications

If there are any clarifications required, developers contact the PM who contacts the solution manager who contacts the business and all the way back introducing a latency of nearly 24 hours and massive email chains for any clarifications

By the time the tech spec is made, nearly 1 month has passed in back and forth

Now, 2 weeks go to development while the test writes test cases

Code is dropped formally to test, test starts raising bugs. Even if there is 1 root cause for 10 different issues, and its an easily fixed one, developers are not allowed to give fresh code to test for the next 1 week. After 2-3 such drops to test the code is given to the ops team as a "golden drop"

( 2 months passed from the beginning)

Ops team will now deploy the code in a staging environment. If it runs stable for a week, it will be promoted to UAT and after 2 weeks of that it will be promoted to prod. If there are any bugs found here, well, applying for a visa requires less paperwork

This entire process is followed even if a single SSRS report is to be released.

How do other companies process such requirements?  I'm wondering why, the business cannot just drop the requirements to developers, developers build and deploy to UAT themselves, expose it to the business who raise functional bugs and after fixing those promote to prod. (even for more complex stuff)

答:
How do other companies process such requirements?

Differently.

Different businesses have different processes because they have different needs, different risk tolerance, different resources...

I'm wondering why, the business cannot just drop the requirements to developers

Often, because it's difficult/impossible/expensive to find developers that are good at being developers and also good at talking to business people. The opposite is of course a problem too - finding business people who are good at business and can talk to developers.

developers build and deploy to UAT themselves

Often, because letting Ops do it frees the developers to focus on the code. It also let's the Ops people learn how the system works and maintain/support it. That support can make or break a product.

But all of this is (mostly) devil's advocate on my part. Too often, process is implemented by managers who don't trust their employees to do their jobs rather than documented (and continually re-evaluated) by managers so that new employees can more easily learn how to do what everyone else does.


答:
I always have a utils package of some sort, even in Java, but my PHP utils collection is the most re-used. There are so many good libraries in Java, that I either already have a library included in the project or need to just design a few missing utils on my own.  PHP libraries tend to do too much for me to want to include them in my projects.

I kind of like this function for PHP, refined with help on StackOverflow...

```<code>function getValueFromDotKey(&$context, $name) {
    $pieces = explode('.', $name);
    foreach ($pieces as $piece) {
        if (!is_array($context) || !array_key_exists($piece, $context)) {
            // error occurred
            return null;
        }
        $context = &$context[$piece];
    }
    return $context;
}
</code>```

It is similar to Apache's BeanUtils for Java, and I use it for a similar purpose, giving form elements in a template language a single key that can get/set a nested value in a source array:

```<code>$source = array('a' => array('b' => 5));

$val = getValueFromDotKey($source, 'a.b');
</code>```

Of course, being PHP, I wanted to keep the method as lightweight as possible so it isn't quite as featureful as BeanUtils ;)

ContentLicense: CC BY-SA 2.5

Title: Do you have your own 'misc utils' library? What part are you most proud of?

Body:
I know that many of us maintain our own little personal library with tools and utilities that we use often.

I've had mine since I was 16 years old so it has grown to quite a considerable size. Some of the stuff I've written has since been added to the framework. I wrote my own little implementation of expression trees for use with genetic algorithms long before LINQ, which I quite like and was proud of at the time - of course its pretty useless now. But recently I have been going through it and upgrading to .NET 4.0 and re-kindled an interest.

So I'm curious as to what you use your library for. Maybe we could get some cool ideas going for useful little snippets and share them amongst ourselves.

So my questions are:

Do you have a miscellaneous utility library?

Which part are you most proud of and why?

Give an example of code if you like :-)

Body:
I always have a utils package of some sort, even in Java, but my PHP utils collection is the most re-used. There are so many good libraries in Java, that I either already have a library included in the project or need to just design a few missing utils on my own.  PHP libraries tend to do too much for me to want to include them in my projects.

I kind of like this function for PHP, refined with help on StackOverflow...

function getValueFromDotKey(&$context, $name) {
    $pieces = explode('.', $name);
    foreach ($pieces as $piece) {
        if (!is_array($context) || !array_key_exists($piece, $context)) {
            // error occurred
            return null;
        }
        $context = &$context[$piece];
    }
    return $context;
}

It is similar to Apache's BeanUtils for Java, and I use it for a similar purpose, giving form elements in a template language a single key that can get/set a nested value in a source array:

$source = array('a' => array('b' => 5));

$val = getValueFromDotKey($source, 'a.b');

Of course, being PHP, I wanted to keep the method as lightweight as possible so it isn't quite as featureful as BeanUtils ;)


问:
Is the Entity Component System architecture object oriented by definition?
Is the Entity Component System architecture object oriented, by definition?  It seems more procedural or functional to me.  My opinion is that it doesn't prevent you from implementing it in an OO language, but it would not be idiomatic to do so in a staunchly OO way.

It seems like ECS separates data (E & C) from behavior (S). As evidence:

The idea is to have no game methods embedded in the entity.

And:

The component consists of a minimal set of data needed for a specific purpose

Systems are single purpose functions that take a set of entities which have a specific component

I think this is not object oriented because a big part of being object oriented is combining your data and behavior together.  As evidence:

In contrast, the object-oriented approach encourages the programmer to place data where it is not directly accessible by the rest of the program. Instead, the data is accessed by calling specially written functions, commonly called methods, which are bundled in with the data.

ECS, on the other hand, seems to be all about separating your data from your behavior.

答:
Entity component systems (ECSs) can be programmed in an OOP or functional manner depending on how the system is defined.

OOP way:

I have worked on games where an entity was an object composed of various components. The entity has an update function which modifies the object in place by calling update on all its components in turn. This is clearly OOP in style - behaviour is linked to data, and the data is mutable. Entities are objects with constructors/destructors/updates.

More functional way:

An alternative is for the entity to be data without any methods. This entity may exist in its own right or simply be an id which is linked to various components. This way it is possible (but not commonly done) to be fully functional and have immutable entities and pure systems which generate new component states.

It seems (from personal experience) that the latter way is gaining more traction and for good reason. Separating entity data from behaviour results in more flexible and reusable code (imo). In particular, using systems to update components/entities in batches can be more performant, and completely avoids the complexities of inter-entity messaging which plague many OOP ECSs.

TLDR: You can do it either way, but I would argue that the benefits of good entity component systems derive from their more functional nature.

ContentLicense: CC BY-SA 4.0


答:
•The "Business" works with a solution manager to build a Business
  Requirement document

Your team must be ready, willing and able to work with the "Business" and create requirements. They may want to keep the same document formats and probably interpret those as some type of agreement or contract between them and the developers. If your development team has a preference to some other type of documentation (sticky notes on a board is still documentation, just not traditional), you may be able to convince the business to accept this. Ideally it is something everyone understands and agrees on.

Now all you have to do is decide what all these other people are going to do. Whether you realize it or not; you not only declared war, but fired the first shot.

Title: Why is a software development life-cycle so inefficient?

Body:
Currently the software development lifecycle followed in the IT company I work at is:

The "Business" works with a solution manager to build a Business Requirement document

The solution manager works with the Program manager to build a Functional Spec

The PM works with the engineering lead to develop a release plan and with the engineering team to develop technical specifications

If there are any clarifications required, developers contact the PM who contacts the solution manager who contacts the business and all the way back introducing a latency of nearly 24 hours and massive email chains for any clarifications

By the time the tech spec is made, nearly 1 month has passed in back and forth

Now, 2 weeks go to development while the test writes test cases

Code is dropped formally to test, test starts raising bugs. Even if there is 1 root cause for 10 different issues, and its an easily fixed one, developers are not allowed to give fresh code to test for the next 1 week. After 2-3 such drops to test the code is given to the ops team as a "golden drop"

( 2 months passed from the beginning)

Ops team will now deploy the code in a staging environment. If it runs stable for a week, it will be promoted to UAT and after 2 weeks of that it will be promoted to prod. If there are any bugs found here, well, applying for a visa requires less paperwork

This entire process is followed even if a single SSRS report is to be released.

How do other companies process such requirements?  I'm wondering why, the business cannot just drop the requirements to developers, developers build and deploy to UAT themselves, expose it to the business who raise functional bugs and after fixing those promote to prod. (even for more complex stuff)


问:
Purpose of building file using Make
I am trying to understand what is the purpose of making files using commands such as cmake .. and make, I have tried looking online but there is no concise explanation on its purpose that i could find.

Also, is it necessary to use make on a project folder that is not built in order to use the source (C++) in other projects.

I know this may be a simplistic question, but as i am fairly new to building files, any information or references would be much appreciated.

答:
make is sort of a batch process for compiling/linking/building, etc.  You could do all of these steps manually, or you can just use make.

More info here: http://c.learncodethehardway.org/book/ex2.html and http://www.gnu.org/software/make/

Title: Purpose of building file using Make


答:
Restructuring everything completely during the migration will probably mean you introduce a high risk of breaking the existing build processes. I would think twice before going that route. The much better alternative is to make 1:1 migration first, and change the structure afterwards, step by step. Any modern versioning system will record the structural changes for you, so if you break something during restructuring, the versioning system will be your safe net.

(Ok, maybe you can leave out the code which is not in use and not under maintenance any more, assumed you can isolate it easily from the other parts. But if not, better migrate it first and delete it afterwards, just like above).

And what about your revision history? When we migrated some years ago from CVS to SVN, keeping the history intact was one of our major requirements - and I can tell you that we never regretted it. So if you see a chance to migrate the history, take it. Changing the structure during the migration will most probably not allow this, but just "dumping all the data and import it into the new system" will not keep the history, either. The better alternative is to create some script (or search the web if someone has already done this for your) which reproduces all commits from the old system into the new one.

One word about the idea of structuring "by language". My team has inherited such repository from another group of developers who thought that structuring "by language" was a good idea. From that experience, I can tell you that its definitely not. At least, not if you want to keep things together which belong together, and the life time of your software is several years. That's because there will be sometimes changes in the language for a subsystem, or one of your subsystems will be extended by another tool or program written in a different language than the rest. And if the folder hierarchy in your production environment will have some similarities to the folder hierachy of your development environment, then structuring "by language" is counter productive, since from the user's point of view the language in which a certain product is written does not really matter.

Title: Code base migration - old versioning system to modern

Body:
Our current code base is contained in a versioning system that is old and outdated (Visual Sourcesafe 5.0, mid 1990's), and contains a mix of packages that are no longer used, ones that are being used but no longer updated, and newer code. It is also a mix of 4 languages, and includes libraries for some of our systems (Such as Dialogic, Sun Tzu {clipper}) implementations. This breaks down into the following categories:

Legacy code - No longer used (Systems that have been retired or replaced, etc)

Legacy code - In current use (No intentions for upgrades or minor bug fixes, only major fixes if needed)

Current code - In current use, and will be used for future versions/development

Support libraries - For both legacy and current code (Some of the legacy libraries are no longer available as well)

We would like to migrate this to a newer versioning system as we will be adding more developers, and expanding the reach to include remote programmers.

When migrating, how do you structure it? Do you just perform a dump of all the data and then import it into the new system, or do you segregate according to type before you bring it into the new system? Do you set up a separate area for libraries, or keep them with the relevant packages? Do you separate by language, system, both? A general outline and methodology is fine, it doesn't need to be broken down to individual program level.


问:
How to keep AST for feature access?
Consider such code (let's say it is C++)

```<code>Foo::Bar.get().X
</code>```

How one should keep the AST for this -- as "tree" with root at left Foo(Bar(get(X)), or with root at right (((Foo)Bar)get)X? Or maybe as a flat structure (list)?

The first one seems more convenient when resolving names, the second when working with it as expression.

I set tag parsing but I am asking from semantic analysis POV really (there is no such tag).

答:
Your example gets the attribute X of the result of calling method bar() on the result of getting static member Bar of the result of evaluating the name Foo. As a tree:</p>

```<code>GetAttribute
+-CallMethod
| +-GetStaticMember
| | +-Foo
| | +-Bar
| +-bar
| +-[] // parameters
+-X
</code>```

Title: How to keep AST for feature access?

Body:
Your example gets the attribute X of the result of calling method bar() on the result of getting static member Bar of the result of evaluating the name Foo. As a tree:

GetAttribute
+-CallMethod
| +-GetStaticMember
| | +-Foo
| | +-Bar
| +-bar
| +-[] // parameters
+-X


问:
Should custom data elements be stored as XML or database entries?
There are a ton of questions like this, but they are mostly very generalized, so I'd like to get some views on my specific usage.

General:

I'm building a new project on my own in Django.  It's focus will be on small businesses.  I'd like to make it somewhat customizble for my clients so they can add to their customer/invoice/employee/whatever items.  My models would reflect boilerplate items that all ModelX might have. For example:

first name

last name

email

address

...

Then my user's would be able to add fields for whatever data they might like.  I'm still in design phase and am building this myself, so I've got some options.

Working on...

Right now the 'extra items' models have a FK to the generic model (Customer  and CustomerDataPoints for example).  All values in the extra data points are stored as char and will be coerced/parced into their actual format at view building.  In this build the user could theoretically add whatever values they want, group them in sets and generally access them at will from the views relavent to that model.

Pros: Low storage overhead, very extensible, searchable

Cons: More sql joins

My other option is to use some type of markup, or key-value pairing stored directly onto the boilerplate models.  This coul essentially just be any low-overhead method weather XML or literal strings.  The view and form generated from the stored data would be taking control of validation and reoganizing on updates.  Then it would just dump the data back in as a char/blob/whatever.

Something like:

```<code><datapoint type='char' value='something' required='true' />
<datapoint type='date' value='01/01/2001' required='false' />
...
</code>```

Pros: No joins needed, Updates for validation and views are decoupled from data

Cons: Much higher storage overhead, limited capacity to search on extra content

So my question is:

If you didn't live in the contraints impose by your company what method would you use? Why?  What benefits or pitfalls do you see down the road for me as a small business trying to help other small businesses?

Just to clarify, I am not asking about custom UI elements, those I can handle with forms and template snippets.  I'm asking primarily about data storage and retreival of non standardized data relative to a boilerplate model.

答:
I think your first model would be perfect if used with a No SQL database. I'm not an expert on No SQL DBs but as I understand it I think this would be a scenario where you might be able to eliminate the extra joins.
Here's the page on No SQL DBs from Wikipedia. I'd take a read and see if that's a viable option for you.</p>


答:
If your program consists of a only a few files that can be built with a simple command line invoking the compiler, then make may be superfluous. However, in large projects you may have dozens of files that require complex compiler and linker options, then make or a similar tool becomes a necessity. Consider, if you have a project with 50 C++ source files, and you change just one of them, you don't want to have re-compile all of them. You'd much rather  just re-compile the one file that changed and re-link with the existing object files for the rest of them. Tools like make automatically take care of chores like that.

The make language has a number of warts, and I don't think it is anybody's favorite programming language. It is also very tricky to write a makefile that will work on multiple platforms, say Linux, OS X, and Windows. Tools like cmake can be used to generate makefiles that will work on multiple platforms, and lets you describe the process of building your software in a language that is arguably less ugly than make proper.

Title: Purpose of building file using Make

Body:
I am trying to understand what is the purpose of making files using commands such as cmake .. and make, I have tried looking online but there is no concise explanation on its purpose that i could find.

Also, is it necessary to use make on a project folder that is not built in order to use the source (C++) in other projects.

I know this may be a simplistic question, but as i am fairly new to building files, any information or references would be much appreciated.


问:
Monitoring C++ applications
We're implementing a new centralized monitoring solution (Zenoss). Incorporating servers, networking, and Java programs is straightforward with SNMP and JMX.

The question, however, is what are the best practices for monitoring and managing custom C++ applications in large, heterogenous (Solaris x86, RHEL Linux, Windows) environments?

Possibilities I see are:

Net SNMP

Advantages

single, central daemon on each server

well-known standard

easy integration into monitoring solutions

we run Net SNMP daemons on our servers already

Disadvantages:

complex implementation (MIBs, Net SNMP library)

new technology to introduce for the C++ developers

rsyslog

Advantages

single, central daemon on each server

well-known standard

unknown integration into monitoring solutions (I know they can do alerts based on text, but how well would it work for sending telemetry like memory usage, queue depths, thread capacity, etc)

simple implementation

Disadvantages:

possible integration issues

somewhat new technology for C++ developers

possible porting issues if we switch monitoring vendors

probably involves coming up with an ad-hoc communication protocol (or using RFC5424 structured data; I don't know if Zenoss supports that without custom Zenpack coding)

Embedded JMX (embed a JVM and use JNI)

Advantages

consistent management interface for both Java and C++

well-known standard

easy integration into monitoring solutions

somewhat simple implementation (we already do this today for other purposes)

Disadvantages:

complexity (JNI, thunking layer between native C++ and Java, basically writing the management code twice)

possible stability problems

requires a JVM in each process, using considerably more memory

JMX is new technology for C++ developers

each process has it's own JMX port (we run a lot of processes on each machine)

Local JMX daemon, processes connect to it

Advantages

single, central daemon on each server

consistent management interface for both Java and C++

well-known standard

easy integration into monitoring solutions

Disadvantages:

complexity (basically writing the management code twice)

need to find or write such a daemon

need a protocol between the JMX daemon and the C++ process

JMX is new technology for C++ developers

CodeMesh JunC++ion

Advantages

consistent management interface for both Java and C++

well-known standard

easy integration into monitoring solutions

single, central daemon on each server when run in shared JVM mode

somewhat simple implementation (requires code generation)

Disadvantages:

complexity (code generation, requires a GUI and several rounds of tweaking to produce the proxied code)

possible JNI stability problems

requires a JVM in each process, using considerably more memory (in embedded mode)

Does not support Solaris x86 (deal breaker)

Even if it did support Solaris x86, there are possible compiler compatibility issues (we use an odd combination of STLPort and Forte on Solaris

each process has it's own JMX port when run in embedded mode (we run a lot of processes on each machine)

possibly precludes a shared JMX server for non-C++ processes (?)

Is there some reasonably standardized, simple solution I'm missing?

Given no other reasonable solutions, which of these solutions is typically used for custom C++ programs?

My gut feel is that Net SNMP is how people do this, but I'd like other's input and experience before I make a decision.

答:
I'm picking up on this as we recently went through the very same process like you:
We were looking for a lightweight, non-blocking, open-source solution which allows exposing and subsequent remote monitoring of metrics from within C/C++ services (we have around ~3000).

SNMP came closest but the integration into the source and the monitoring system is a pain and not suitable for our real-time processes.

In the end, we decided to develop a new solution called CMX which uses shared memory technology and made it open-source. You can check it out here :
www.cern.ch/cmx.

Title: Monitoring C++ applications


问:
Why is the function called lseek(), not seek()?
The C function for seeking in a file is called lseek(). Why ins't it called just seek()?</p>

答:
It appears that the 'l' in lseek means 'long', to differentiate from an old seek function in version 2 of AT&T Unix. This is an anachronism before the off_t type was introduced.

Supporting references:

http://infohost.nmt.edu/~eweiss/222_book/222_book/0201433079/ch03lev1sec6.html indicates:

The character l in the name lseek means "long integer." Before the
  introduction of the off_t data type, the offset argument and the
  return value were long integers. lseek was introduced with Version 7
  when long integers were added to C. (Similar functionality was
  provided in Version 6 by the functions seek and tell.)

As noted at the foot of http://www.daemon-systems.org/man/lseek.2.html

```<code> A seek() function appeared in Version 2 AT&T UNIX, later renamed into
 lseek() for ``long seek'' due to a larger offset argument type.
</code>```

Body:
It appears that the 'l' in lseek means 'long', to differentiate from an old seek function in version 2 of AT&T Unix. This is an anachronism before the off_t type was introduced.

Supporting references:

http://infohost.nmt.edu/~eweiss/222_book/222_book/0201433079/ch03lev1sec6.html indicates:

The character l in the name lseek means "long integer." Before the
  introduction of the off_t data type, the offset argument and the
  return value were long integers. lseek was introduced with Version 7
  when long integers were added to C. (Similar functionality was
  provided in Version 6 by the functions seek and tell.)

As noted at the foot of http://www.daemon-systems.org/man/lseek.2.html

 A seek() function appeared in Version 2 AT&T UNIX, later renamed into
 lseek() for ``long seek'' due to a larger offset argument type.


问:
Demonstration of garbage collection being faster than manual memory management
I've read in many places (heck, I've even written so myself) that garbage collection could (theoretically) be faster than manual memory management.

However, showing is a lot harder to come by than telling.

I have never actually seen any piece of code that demonstrates this effect in action.

Does anyone have (or know where I can find) code that demonstrates this performance advantage?

答: Faster is dubious. However, it can be ultra-fast, imperceptible, or faster if it's hardware supported. There were designs like that for LISP machines a long time ago. One built the GC into the memory subsystem of the hardware as such that main CPU didn't know it was there. Like many later designs, the GC ran concurrently with the main processor with little or no need for pauses. A more modern design is Azul Systems Vega 3 machines that run Java code way faster than JVM's using purpose-built processors and a pauseless GC. Google them if you want to know how fast GC (or Java) can be. </p>


答:
First of all, your application should never care about your data storage. Data storage, e.g., a database, is a detail. What your application should care about is its entities. Where these entities get there data from is irrelevant to the application. All your application should do is use an entity gateway to manipulate data, i.e., an interface which has methods for the entities. For example, an OrderGateway has methods like PutOrder, GetOrderById. The interfaces are implemented in the data access layer. Following this architecture, you can then implement whatever storage mechanisms you would like—database, file system, web service, etc. This approach leaves your application open for expansion.

Start with the most simple, which is usually basic file system access. Then, if your customers or users require it, implement other means of data storage. Don’t build things on vague assumptions like, “perhaps my customers would like to work on an SQL Server, or perhaps my users want to search on extra content”—you aren’t going to need it (yet).

Go with your XML-approach (or some similar file system-based solution). Worrying about storage overhead? Well, has it proved itself to be a problem? If not, then it isn’t an issue. Worrying about search capacity? Same thing. If it becomes a problem, then optimize accordingly. Early optimization and anticipated problems must be supported by strong evidence before they can be justified in software development.

Title: Should custom data elements be stored as XML or database entries?

Body:
There are a ton of questions like this, but they are mostly very generalized, so I'd like to get some views on my specific usage.

General:

I'm building a new project on my own in Django.  It's focus will be on small businesses.  I'd like to make it somewhat customizble for my clients so they can add to their customer/invoice/employee/whatever items.  My models would reflect boilerplate items that all ModelX might have. For example:

first name

last name

email

address

...

Then my user's would be able to add fields for whatever data they might like.  I'm still in design phase and am building this myself, so I've got some options.

Working on...

Right now the 'extra items' models have a FK to the generic model (Customer  and CustomerDataPoints for example).  All values in the extra data points are stored as char and will be coerced/parced into their actual format at view building.  In this build the user could theoretically add whatever values they want, group them in sets and generally access them at will from the views relavent to that model.

Pros: Low storage overhead, very extensible, searchable

Cons: More sql joins

My other option is to use some type of markup, or key-value pairing stored directly onto the boilerplate models.  This coul essentially just be any low-overhead method weather XML or literal strings.  The view and form generated from the stored data would be taking control of validation and reoganizing on updates.  Then it would just dump the data back in as a char/blob/whatever.

Something like:

```<code><datapoint type='char' value='something' required='true' />
<datapoint type='date' value='01/01/2001' required='false' />
...
</code>```

Pros: No joins needed, Updates for validation and views are decoupled from data

Cons: Much higher storage overhead, limited capacity to search on extra content

So my question is:

If you didn't live in the contraints impose by your company what method would you use? Why?  What benefits or pitfalls do you see down the road for me as a small business trying to help other small businesses?

Just to clarify, I am not asking about custom UI elements, those I can handle with forms and template snippets.  I'm asking primarily about data storage and retreival of non standardized data relative to a boilerplate model.


答: Short answer: Other companies that don’t use the waterfall approach that you’re describing to build IT solutions often use an Agile approach, e.g., Scrum or eXtreme Programming.</p>


答:
I think your custom data elements would fall under #3 of this answer to a similar question:

Values are likely to change frequently and/or meant to be edited by non-developers, but yet this changes will not affect the logic - database with ORM or at least key-value storage with some user-friendly interface.

You probably don't want to be responsible for adding the custom data points for each of your users, so making it easy for them to do themselves should be a pretty high priority. Asking a small-business owner to add custom data points to (essentially) a configuration file is going to turn a lot of people off, and if you're already familiar with working with databases in code, storing them there will make it much easier to build a UI for them.

As long as your tables are indexed well - the Customer foreign key on CustomerDataPoints is indexed - the extra joins won't add an appreciable amount of performance overhead.


问:
Strategy to store/average logs of pings
I'm developing a site to monitor web services. The most basic type of check is sending a ping, storing the response time in a CheckLog object. By default, PingCheck objects are triggered every minute, so in one hour you get 60 CheckLogs and in one day you get 1440 CheckLogs.

That's a lot of them, I don't need to store such level of detail, so I've set a up collapsing mechanism that periodically takes the uncollapsed CheckLogs older than 24h and collapses (averages) them in intervals of 30 minutes. So, if you have 360 CheckLogs that have been saved from 0:00 to 6:00, after collapsing you retain just 12 of them. The problem.. well, is this:

After averaging the response times, the graph changes drastically. What can I do to improve this? Guess one option could be narrowing the interval duration to 15 min.

I've seen the graphs at the GitHub status page and they do not seem to suffer from this problem.

I'd appreciate any kind of information you could give me about this area.

答: As @Uri Agassi pointed out, there looks to be quite a bit of variance. To that end, I think you mostly care about the "shape" of the curve, rather than the actual points themselves. So then the question becomes different: how do I preserve the "shape" to within a certain tolerance? Fortunately there are good algorithms around this type of question, one of the standards being this one which you can use to reduce the number of polylines in your graph. The unfortunate reality here, though, is that the amount of storage will no longer be deterministic because it is now based on variance - unless you pick a different polyline reduction technique.</p>

Title: Strategy to store/average logs of pings


问:
Is my work on a developer test being taken advantage of?
I am looking for a job and have applied to a number of positions. One employer responded. I had a pretty lengthy phone interview (perhaps an hour +) and they then set me up with a developer test. I was told that the test was estimated to take between 6 and 8 hours and that, provided the results met with their approval, I would be paid for my work.

That gave me some pause, but I endeavored. The developer test took place on a VM accessed via RDP. The task was to implement a search page in a web project that requests data from the server, displays it on the screen in a table, has a pretty complicated search filtering scheme (there are about 15 statuses and when sending the search to the server you can search by these statuses) in addition to the string/field search. Additionally, they want SVG icons to change color on certain data values, and some data represented differently than how it's structured in the database.

Loooong story short, this took a heck of a lot longer than 6-8 hours. Much of it was due to the very poor VM that I was running on (Visual Studio 2013 took 10 minutes to load, and another 15 minutes to open the 3 GB ginormous solution).

I was told that after completing the test I should commit my changes to source control...  Hmm, OK. I followed directions. And after committing the changes, I was emailed a response. The SVGs weren't colored right, there was a bug in this edge-case, there was an occasional problem with this other thing that I never experienced, etc. So I am 13-14 hours into this thing now, and I have to do bug fixes. I do them, and the employer comes back with more bug fix requests.

All my work is apparently going into a production application. I noticed a few anomalies in the code where it looked like others had coded all of one functionality but hadn't touched anything else.

Am I just being used for cheap labor? Even if they pay me the promised 50 dollars an hour for 6 hours, I have committed about 18 hours to this thing now. If I bug fix all of the stuff they keep coming up with, I will have worked at least 16 hours for free.

I have taken a number of developer tests, but I have never taken one during which I worked on code destined for production. I have never taken a test where I implemented a feature that was in the pipeline for development, and I have never taken one that took 4 rounds and a total of 20+ hours. I get the impression they are using their developer test to field some of the functionality on the cheap.

Do I have the wrong impression? And is this testing protocol appropriate?

答:
People with more career experience may be better able to answer this question, but I personally would not be very comfortable with a 20+ hour dev test. It sounds like they are using the interview to get work tasks completed.

I am assuming that you have not signed any legal documents concerning ownership of the code. So I would wait until they reviewed the code and accepted or denied it. Then if they accepted it I would ask to get paid for the full time, 20+ hours. I'm not sure that I would take payment for just the six hours that was originally suggested. If this is going to go into production, then they will need to straighten out code ownership.

At the very least, discussing payment for the code should help you decide whether you would want to accept an offer. I wouldn't want to accept an offer if they thought only paying you for six hours was fair.


答:
Many interviews are followed by tests. Those tests are needed to ensure that you actually have the required skills and to give a better view of a few things which are difficult to test during the interview itself (such as do you apply style rules to your code).

This being said, a test is a test.

It doesn't need to be long. There is not much you can see after eight hours of coding that you cannot see after thirty minutes. More importantly, code written during the test should then be reviewed, line per line, which takes an important amount of time. It is not unusual to spend more than two hours to review the test code written during half an hour.

It shouldn't deal with an existent code base. Understanding the code base of a medium-scale product may take days or weeks (or months or years depending on the code quality and technical debt). Intellectual property may also be an issue (unless the code is open sourced).

When the goal is to test how the candidate is able to maintain the existent code base, the test can be done on a small (500-600 LOC) fictional code base written specifically for the tests.

It doesn't have to be a request to develop a real-life app or feature. It can be a completely useless piece of code, written with the sole intention to show that you've understood the problem and found an elegant way to solve it.

It doesn't have to be perfect. There are bugs? That's fine. Take a note of them for a further interview with the candidate; it can be an excellent opportunity to see how the candidate reacts in this situation.

It doesn't have to be done through RDC on a VM, unless you don't have Visual Studio yourself. If the goal is to see your coding and problem solving skills, it doesn't matter where do you do the exercise.

It's out of question for the code written during this test to end up in the version control of the company. Why would they pollute their version control with something written by a candidate?

To conclude, when you're asked to spend dozens of hours writing production code, solving bugs, and committing your work to the version control of the company:

Either they are just using you to implement features for free,

Or they really don't understand how to do an interview.

In both cases, look for a better place to work.

Title: Is my work on a developer test being taken advantage of?

Body:
I am looking for a job and have applied to a number of positions. One employer responded. I had a pretty lengthy phone interview (perhaps an hour +) and they then set me up with a developer test. I was told that the test was estimated to take between 6 and 8 hours and that, provided the results met with their approval, I would be paid for my work.

That gave me some pause, but I endeavored. The developer test took place on a VM accessed via RDP. The task was to implement a search page in a web project that requests data from the server, displays it on the screen in a table, has a pretty complicated search filtering scheme (there are about 15 statuses and when sending the search to the server you can search by these statuses) in addition to the string/field search. Additionally, they want SVG icons to change color on certain data values, and some data represented differently than how it's structured in the database.

Loooong story short, this took a heck of a lot longer than 6-8 hours. Much of it was due to the very poor VM that I was running on (Visual Studio 2013 took 10 minutes to load, and another 15 minutes to open the 3 GB ginormous solution).

I was told that after completing the test I should commit my changes to source control...  Hmm, OK. I followed directions. And after committing the changes, I was emailed a response. The SVGs weren't colored right, there was a bug in this edge-case, there was an occasional problem with this other thing that I never experienced, etc. So I am 13-14 hours into this thing now, and I have to do bug fixes. I do them, and the employer comes back with more bug fix requests.

All my work is apparently going into a production application. I noticed a few anomalies in the code where it looked like others had coded all of one functionality but hadn't touched anything else.

Am I just being used for cheap labor? Even if they pay me the promised 50 dollars an hour for 6 hours, I have committed about 18 hours to this thing now. If I bug fix all of the stuff they keep coming up with, I will have worked at least 16 hours for free.

I have taken a number of developer tests, but I have never taken one during which I worked on code destined for production. I have never taken a test where I implemented a feature that was in the pipeline for development, and I have never taken one that took 4 rounds and a total of 20+ hours. I get the impression they are using their developer test to field some of the functionality on the cheap.

Do I have the wrong impression? And is this testing protocol appropriate?


问:
Modular Web App Network Architecture
Assuming that I am dealing with dedicated physical servers or VPSs, is it conceivable and does it make sense to have distinct servers setup with the following roles to host a web application?

Reverse Proxy

Web server

Application server

Database server

Specific points of interest:

I am confused how to even separate the web and application servers.  My understanding was that such 3-tier architectures were feasible.

It is unclear to me if the app server would reside directly between the web and database server, or if the web server could directly interact with the database as well.  The app server could either do the computational heavy-lifting on behalf of the app server or it could do heavy-lifting plus control all of the business logic (as implied in the diagram above, thus denying the web server of direct database access).

I am also unsure what role the reverse proxy (ex. nginx) could and should fulfill as a web server, given the above mentioned setup.  I know that nginx has web server features.  But I do not know if it makes sense to have the reverse proxy be its own VPS, given that the web server–in theory–would be separate from the app server.

答:
Remember one key rule:

Don't worry too much about getting the architecture scalable and all in the first pass. You might need to change it several times over as your application evolves as a result of more users/traffic/features/etc.

Just get down to shipping something so that REAL users can have a go, and act on their feedback. Don't plan ahead too much because no matter what to plan, it is bound to change.

As for the quesrion, the reverse proxy and the web server can generally be the same application -- usually nginx to begin with. Then as you scale out, varnish can be a more powerful reverse proxy.

Web servers like nginx never talk to the database directly. Application servers sit between the two.

-- UPDATE --

The above does NOT mean that we do not need to think about architecture at all. It means not worrying about getting every last architectural detail right from the start.

However, we SHOULD think about the high level system shape. Interactions between components is something that must be well thought out before beginning the concrete implementation.

Title: Modular Web App Network Architecture


问:
Keep user and user profile in different tables?
I have seen in a couple of projects that developers prefer to keep essential user info in one table (email/login, password hash, screen name) and rest of the non essential user profile in another (creation date, country, etc). By non-essential I mean that this data is needed only occasionally. Obvious benefit is that if you are using ORM querying less fields is obviously good. But then you can have two entities mapped to same table and this will save you from querying stuff you don't need (while being more convenient). Does anybody know any other advantage of keeping these things in two tables?</p>

答:
It depends on the size and requirements of your project.

I can see one way in which data about users can be divided into two sets, with different purposes and thus requirements:

Identity data: user name, password hash, email address, last login time, etc.

User profile data, which includes users preferences, latest activity, status updates etc.

Note that there are some attributes about the user that can fall in either category (e.g. user's date of birth). The difference between these two sets though is that the first one is tightly controlled and only through certain workflows can it be modified. For example, changing a password may require providing existing password, changing the email may require verification of email, and it would be used in the case the user forgets the password.

Preferences do not require such ACLs, and could theoretically be modified by the user or another application so long as user consents to it. The stakes are low if an application maliciously or due to a bug corrupts the data or attempts to modify it (assuming other security measures are taken.) However, it would usually be disastrous if any of the user name, password or email could be modified since they can either be used to assume user's identity or deny service or cause support costs etc. for the admin.

Thus, usually the data is stored in two types of systems:

Identity data will typically go in a directory or an IAM solution.

Preference data will end up in a database.

Having said that, in practice, people will violate these rules and use one or the other (e.g. SQL server behind ASP.NET membership provider).

As the identity data becomes larger or the organization that uses it becomes larger, different types of problems creep in. For example, in the case of directory, it will attempt replicate the password changes immediately to all servers in a multi-server environment. However, user preference only require eventual consistency. (FYI: Both of these are different optimizations of CAPS theorem.)

Finally, directories (esp. the online/cloud directories) will also issue access tokens for other resources using protocols such as OAUTH (e.g. consider Facebook, Google, Microsoft Account, ADFS), whereas a database has no such need. A database will support quite complex joins and query structure, which directory does not need.

For more details, a few searches on identity directory vs database would help.

It eventually comes down to what your scenarios are and expected to be in the future, including integration with any third parties (and what they are using). If it's a well-contained project and you're confident that you can secure the user identity data and authenticate correctly, then you could go for database. Otherwise, it might be worth investigating an identity directory.

If you go for DB, IMO, using one DB vs two would eventually come down to access control, both for users and applications.


答:
I would never participate in a code test of this nature. I have taken many code tests and done many code projects. I certainly wouldn't check code into someone else's repository under any circumstance. If they don't know what they need to know after a 4 hour sample with some minor bug correction in a pair-programming session, then they won't ever know.

Going into a test, you should know and make clear a few things up front:

It should be agreed upon and known that any work produced during the test may not be used for any purpose other than determining your skill at the required tasks.

A code test should not last more than 4 hours.

You are not an employee of the company, so any suggestion that you might be paid for code produced is preposterous. Insist on a written contract of payment if there is even a hint of this.

Set specific limits on the time you will spend on any given part of the test, and then stick to those limits. If you find yourself going over the limits for any reason, consider why you are going over that limit. Is it because of pressure from them? Is it because you've made mistakes? Is it because you've poorly estimated how long something should take to complete?

Stand your ground if you feel you have covered a particular topic. If you've already fixed a bug, and they're asking you to fix a nearly identical bug, say "We've already covered that topic with bug x, perhaps we could move to something else that demonstrates something new."

Under no circumstances should you check anything into a production pipeline. This includes into any kind of development branch that may ultimately lead to a production pipeline. When in doubt, check nothing in. For code tests that are not necessarily in person, I insist that the code be checked into my personal public repository first. This gives me at least some kind of protection from having my work used inappropriately.

Judge them for their behavior every bit as much as they are judging you. If you feel they are not being up front with you, call them on it. If you feel you are being mistreated, speak up.

The company you are interviewing with is also being interviewed by you. If this is how they are treating someone they are interviewing, is this a company you want to work for? I understand that often people have a need for a job and often this need will override some common sense concepts, but this should always be in the forefront of your mind. Don't be afraid to walk out. If it doesn't feel right, follow your instincts and vote with your feet.


问:
Maintaining Two Separate Software Versions From the Same Codebase in Version Control
Let's say that I am writing two different versions of the same software/program/app/script and storing them under version control. The first version is a free "Basic" version, while the second is a paid "Premium" version that takes the codebase of the free version and expands upon it with a few extra value-added features. Any new patches, fixes, or features need to find their way into both versions.

I am currently considering using master and develop branches for the main codebase (free version) along side master-premium and develop-premium branches for the paid version. When a change is made to the free version and merged to the master branch (after thorough testing on develop of course), it gets copied over to the develop-premium branch via the cherry-pick command for more testing and then merged into master-premium.

Is this the best workflow to handle this situation? Are there any potential problems, caveats, or pitfalls to be aware of? Is there a better branching strategy than what I have already come up with?

Your feedback is highly appreciated!

P.S. This is for a PHP script stored in Git, but the answers should apply to any language or VCS.

答:
I strongly recommend not using branches for this purpose. In general, you should consider branches for things that will be (or might be) merged back together again later (or for release branches, where you eventually stop development of one of the branches). In your case, you will never merge your "basic" and "premium" versions together, and they will both be maintained indefinitely, so branches are not appropriate.

Instead, maintain one common version of the source code and use conditional compilation (eg. #ifdef in C/C++, not sure what the equivalent is for PHP) to include or exclude the sections of code that differ between "basic" and "premium".

It looks like PHP might not have such a conditional compilation feature built in, so you could use the C preprocessor (cpp, you probably already have it) to preprocess your common source code and from that, produce a "basic" and a "premium" version without the preprocessor directives. Of course, if you choose to do this you should use make or something similar to automate the process of running the preprocessor.


问:
What is a symbol in Ruby?
I'm totally new to the Ruby world, and I'm a bit confused with the concept of Symbols. What's the difference between Symbols and Variables? Why not just using variables?

Thanks.

答:
Variables and symbols are different things. A variable points to different kinds of data. In Ruby, a symbol is more like a string than a variable.

In Ruby, a string is mutable, whereas a symbol is immutable. That means that only one copy of a symbol needs to be created. Thus, if you have

```<code>x = :my_str
y = :my_str
</code>```

:my_str will only be created once, and x and y point to the same area of memory. On the other hand, if you have

```<code>x = "my_str"
y = "my_str"
</code>```

a string containing my_str will be created twice, and x and y will point to different instances.

As a result, symbols are often used as the equivalent to enums in Ruby, as well as keys to a dictionary (hash).

Title: What is a symbol in Ruby?

Body:
Variables and symbols are different things. A variable points to different kinds of data. In Ruby, a symbol is more like a string than a variable.

In Ruby, a string is mutable, whereas a symbol is immutable. That means that only one copy of a symbol needs to be created. Thus, if you have

x = :my_str
y = :my_str

:my_str will only be created once, and x and y point to the same area of memory. On the other hand, if you have

x = "my_str"
y = "my_str"

a string containing my_str will be created twice, and x and y will point to different instances.

As a result, symbols are often used as the equivalent to enums in Ruby, as well as keys to a dictionary (hash).


问:
Regex to String generation
Let's say that we have a regex and an index i.  If we suppose that the set of strings that match our regex are sorted in a lexicographical order, how could we get the i element of this list?

Edit : I added this simple example for more explanation:

input :

```<code>regex="[a-c]{1,2}";
index=4;
</code>```

In this case the ordered list of strings that this regex matches contains the following  elements:

```<code>a
aa
ab
ac
b
ba
bb
bc
c
ca
cb
cc
</code>```

output :

The 4th element, which is ac

ps: It's known that a given regex could match an infinite number of strings.  This shouldn't have an impact on the process of extracting the element in the given finite index.

答:
I would like to propose this answer just in case it can help you some how.
In case there are some details or some small errors, please leave comment.
I will try to correct.

First look at the minimum finite state automata FSA of the regex

Find the first matching string of the FSA. This can be done by creating an auxiliary FSA in which you remove all paths leading to the reject state, then start walking through the auxiliary FSA. At each transition, choose the first character from the input alphabet to make sure you generate the first matching string.

Now we need to get the second matching string of the original regex, so we modify the auxiliary FSA so that in the final state (before the accep state), the last character input (of the first matching string in 2 above) would not lead to the accept state. This can be done by removing an edge (and a state) or modify the edge leading to the accepting state.

Generate the first matching string of the modified auxiliary FSA, this would give you the second matching string of the original regex

Repeat modifying the auxiliary FSA until you get to the i-th matching string. This would be the i-the matching string of the original regex.

Title: Regex to String generation


问:
Where to Perform Authentication in REST API Server?
I am working on a set of REST APIs that needs to be secured so that only authenticated calls will be performed. There will be multiple web apps to service these APIs. Is there a best-practice approach as to where the authentication should occur?

I have thought of two possible places.

Have each web app perform the authentication by using a shared authentication service. This seems to be in line with tools like Spring Security, which is configured at the web app level.

Protect each web app with a "gateway" for security. In this approach, the web app never receives unauthenticated calls. This seems to be the approach of Apache HTTP Server Authentication. With this approach, would you use Apache or nginx to protect it, or something else in between Apache/nginx and your web app?

For additional reference, the authentication is similar to services like AWS that have a non-secret identifier combined with a shared secret key. I am also considering using HMAC. Also, we are writing the web services in Java using Spring.

Update: To clarify, each request needs to be authenticated with the identifier and secret key. This is similar to how AWS REST requests work.

答:
It is a matter of separation of concerns. In one case, a dedicated identity provider (IDP) does the authentication and provides an access token, which is verified by all web applications before a user (or an application) can execute an operation. In the other case, every web application has to do that authentication (even if they share the same code base).

The implications are significant esp. for consuming code:

If there is one identity provider, a user (or the consuming code) will typically authenticate once, and the access token can then be supplied to as many web applications as needed. This enables single sign on (SSO) to be implemented across multiple domains (consider the case where a user logs in once to Facebook and then don't have to supply credentials for every site that uses FB authentication.)

Since there is one IDP, all the user experience around logins is much easier to manage. For example, a user goes to a web application, it redirects to the IDP to login. The cases such as user forgetting user name, or password, or has some pre-requisite missing on their machine can all be handled by this common service. In the case of each web application doing this, somehow the authentication user experiences have to be exposed through each web application, which is doable, but more difficult to manage.

There're already well-established protocols that you can use for such scenarios (e.g. OAUTH), which means that you can use an external IDP (now or in the future) that supports the same protocol(s) without changing your application much.

Since the specifications for such protocols are standardized, there is higher chance that one would get security right since important parameters are forced into the protocol (although there're still vulnerabilities that one has to manage, e.g. OAUTH is prone to CSRF attacks as an example).

The benefit of each web application doing authentication is:

Initial implementation is usually lower cost, because one does not have to deal with creating access tokens and passing them around and validating them. Once credentials are passed to a web application, it can make an internal call to a common library (or service) to validate the credentials. Usually, web development frameworks already provide some features to accomplish these so the cost is often very low.

If all your apps are on one domain (and you can use cookies for single sign on), it is a far cleaner user experience (but as soon as you have two or more domains, the user experience becomes difficult esp. when there is no single sign on.)

Amazon, Microsoft, Facebook, Google, etc., all use a dedicated service because the pros are far more important for their users and consuming applications than the cost of implementing. In addition, they all have dedicated teams that work on authentication features such as automated password resets, multi-factor authentication, API access, as well as implementing newer protocols and improving existing one, esp. in terms of security.

[Disclaimer: I work at Microsoft in Active Directory and Security group. Opinions are my own, and I have made my best effort to make this response technology agnostic since it is a design question.]


答: We are using 2 separate projects, the Basic one and the Premium one that depends on Basic project. Do not use braches, they are usually used for features.</p>

Title: Maintaining Two Separate Software Versions From the Same Codebase in Version Control

Body:
Let's say that I am writing two different versions of the same software/program/app/script and storing them under version control. The first version is a free "Basic" version, while the second is a paid "Premium" version that takes the codebase of the free version and expands upon it with a few extra value-added features. Any new patches, fixes, or features need to find their way into both versions.

I am currently considering using master and develop branches for the main codebase (free version) along side master-premium and develop-premium branches for the paid version. When a change is made to the free version and merged to the master branch (after thorough testing on develop of course), it gets copied over to the develop-premium branch via the cherry-pick command for more testing and then merged into master-premium.

Is this the best workflow to handle this situation? Are there any potential problems, caveats, or pitfalls to be aware of? Is there a better branching strategy than what I have already come up with?

Your feedback is highly appreciated!

P.S. This is for a PHP script stored in Git, but the answers should apply to any language or VCS.


答:
Until your project is small, it is very easy to build it without any makefiles. But imagine your projects has 200 source files that you've divided in to a number of static and dynamic libraries, then passing the names of 200 files to the compiler in a command prompt would be impossible. Also you can create a batch file which builds your projects. But make files are superior to batch files. Particularly, when you use cmake your source code can be built with any compiler easily. That is why large projects use makefiles.

Note: If you use any convinient IDE like Visual Studio, you'll not often feel the need of writing makefiles, as the IDE makes it easy for you.

Title: Purpose of building file using Make


问:
Use a custom value object or a Guid as an entity identifier in a distributed system?

tl;dr

I've been told that in domain-driven design, an identifier for an entity could be a custom value object, i.e. something other than Guid, string, int, etc. Can this really be advisable in a distributed system?

Long version

I will invent an situation analogous to the one I am currently facing.

Say I have a distributed system in which a central concept is an egg. The system allows you to order eggs and see spending reports and inventory-centric data such as quantity on hand, usage, valuation and what have you. There area variety of services backing these behaviors. And say there is also another app which allows you to compose recipes that link to a particular egg type.

Now egg type is broken down by the species—ostrich, goose, duck, chicken, quail. This is fine and dandy because it means that users don't end up with ostrich eggs when they wanted quail eggs and whatnot. However, we've been getting complaints because jumbo chicken eggs are not even close to equivalent to small ones. The price is different, and they really aren't substitutable in recipes. And here we thought we were doing users a favor by not overwhelming them with too many options.

Currently each of the services (say, OrderSubmitter, EggTypeDefiner, SpendingReportsGenerator, InventoryTracker, RecipeCreator, RecipeTracker, or whatever) are identifying egg types with an industry-standard integer representation the species (let's call it speciesCode). We realize we've goofed up because this change could effect every service.

There are two basic proposed solutions:

Use a predefined identifier type like Guid as the eggTypeID throughout all the services, but make EggTypeDefiner the only service that knows that this maps to a speciesCode and eggSizeCode (and potentially to an isOrganic flag in the future, or whatever).

Use an EggTypeID value object which is a combination of speciesCode and eggSizeCode in every service.

I've proposed the first solution because I'm hoping it better encapsulates the definition of what an egg type is in the EggTypeDefiner and will be more resilient to changes, say if some people now want to differentiate eggs by whether or not they are "organic".

The second solution is being suggested by some people who understand DDD better than I do in the hopes that less enrichment and lookup will be necessary that way, with the justification that in DDD using a value object as an ID is fine. Also, they are saying that EggTypeDefiner is not a domain and EggType is not an entity and as such should not have a Guid for an ID.

However, I'm not sure the second solution is viable. This "value object" is going to have to be serialized into JSON and URLs for GET requests and used with a variety of technologies (C#, JavaScript...) which breaks encapsulation and thus removes any behavior of the identifier value object (is either of the fields optional? etc.) Is this a case where we want to avoid something that would normally be fine in DDD because we are trying to do DDD in a distributed fashion?

Summary

Can it be a good idea to use a custom value object as an identifier in a distributed system (solution #2)?

答:
If I were to simplify it a bit to core requirements, it seems that what the problem comes down to are the following:

Design egg type in such a way that it can be extended in the future to hold additional attributes

Whether such extensions can be backward compatible or not (and whether they should be backward compatible or not)

I would think that the backward compatibility problem is going to be the real issue to solve. Let's walk through the example you provided:

Currently each of the services (say, OrderSubmitter, EggTypeDefiner,
  SpendingReportsGenerator, InventoryTracker, RecipeCreator,
  RecipeTracker, or whatever) are identifying egg types with an
  industry-standard integer representation the species (let's call it
  speciesCode). We realize we've goofed up because this change could
  effect every service.

Stepping away from the implementation for a second, let's say the egg type now included a species code and a size code. Now let's say that a new attribute needed to be added with the assumption that no matter how many attributes are added now, new ones could be added in the future. How should the existing services that are already referencing to existing egg types behave?

One approach, which may be quite acceptable in this case, could be that any new attribute is an 'optional' attribute, which means that there is either a default or a "doesn't matter" flag and existing services continue to operate based on the existing set of attributes. This issue is easier solved if we assume that attributes will only be added (your case, and never modified or removed, which is a more involved versioning issue).

If this approach is acceptable, then I would design an object type (typically a class) that holds these various properties, and ensure that serialization and deserialization into various formats is backward compatible.

For example, let's assume the following (assume pseudocode):

```<code>class EggType
{
    string speciesCode;
    string sizeCode;
}

string SerializeForURL()
{
    return speciesCode + "." + sizeCode;
}
</code>```

Now, let's say we added a new property, called colorCode. But since this is optional, we would assume that an egg type can be created without providing a colorCode. If that's the case, the code should handle the case where colorCode is provided vs where not provided. Let's say if one of the services which uses this egg type places orders for new eggs from the farms, that service will have to be modified to support this, otherwise, new eggs will be random colors regardless of what was ordered. However, let's say there is a service that collects sales data on what is sold, it can simply not aggregate by colorCode (if the code is not yet updated), and not report it up the management chain, until that change is made. This decision of when to support can then rest on respective services.

We could then modify the serializeForUrl() to be something like this:

```<code>string SerializeForURL()
{
    return speciesCode + "." + sizeCode + "." + colorCode;
}
</code>```

In the deserialization method, we could check whether there are two attributes or three. If two, colorCode is not provided, but if three, then colorCode is indeed provided.

It is easier in the case of JSON, because the value for colorCode would be null.

I would prefer this approach over GUIDs any day, for multiple reasons:

Makes it harder to manage change in egg type: It does not allow isolation of these attributes from each other, even though they are really independent of each other. (i.e. if you modify the GUID, all the services need to understand it, even though all it might be doing is adding a property, such as colorCode, which many services may not care about at all.)

All the systems are now dependent on a global list of GUIDs and how they translate into egg type - unnecessary coupling in my opinion.

Easier to debug: programmers are one of the users of the code, and they need to be able to easily read code as well as trace logs, audit data, whatever.

Body:
If I were to simplify it a bit to core requirements, it seems that what the problem comes down to are the following:

Design egg type in such a way that it can be extended in the future to hold additional attributes

Whether such extensions can be backward compatible or not (and whether they should be backward compatible or not)

I would think that the backward compatibility problem is going to be the real issue to solve. Let's walk through the example you provided:

Currently each of the services (say, OrderSubmitter, EggTypeDefiner,
  SpendingReportsGenerator, InventoryTracker, RecipeCreator,
  RecipeTracker, or whatever) are identifying egg types with an
  industry-standard integer representation the species (let's call it
  speciesCode). We realize we've goofed up because this change could
  effect every service.

Stepping away from the implementation for a second, let's say the egg type now included a species code and a size code. Now let's say that a new attribute needed to be added with the assumption that no matter how many attributes are added now, new ones could be added in the future. How should the existing services that are already referencing to existing egg types behave?

One approach, which may be quite acceptable in this case, could be that any new attribute is an 'optional' attribute, which means that there is either a default or a "doesn't matter" flag and existing services continue to operate based on the existing set of attributes. This issue is easier solved if we assume that attributes will only be added (your case, and never modified or removed, which is a more involved versioning issue).

If this approach is acceptable, then I would design an object type (typically a class) that holds these various properties, and ensure that serialization and deserialization into various formats is backward compatible.

For example, let's assume the following (assume pseudocode):

class EggType
{
    string speciesCode;
    string sizeCode;
}

string SerializeForURL()
{
    return speciesCode + "." + sizeCode;
}

Now, let's say we added a new property, called colorCode. But since this is optional, we would assume that an egg type can be created without providing a colorCode. If that's the case, the code should handle the case where colorCode is provided vs where not provided. Let's say if one of the services which uses this egg type places orders for new eggs from the farms, that service will have to be modified to support this, otherwise, new eggs will be random colors regardless of what was ordered. However, let's say there is a service that collects sales data on what is sold, it can simply not aggregate by colorCode (if the code is not yet updated), and not report it up the management chain, until that change is made. This decision of when to support can then rest on respective services.

We could then modify the serializeForUrl() to be something like this:

string SerializeForURL()
{
    return speciesCode + "." + sizeCode + "." + colorCode;
}

In the deserialization method, we could check whether there are two attributes or three. If two, colorCode is not provided, but if three, then colorCode is indeed provided.

It is easier in the case of JSON, because the value for colorCode would be null.

I would prefer this approach over GUIDs any day, for multiple reasons:

Makes it harder to manage change in egg type: It does not allow isolation of these attributes from each other, even though they are really independent of each other. (i.e. if you modify the GUID, all the services need to understand it, even though all it might be doing is adding a property, such as colorCode, which many services may not care about at all.)

All the systems are now dependent on a global list of GUIDs and how they translate into egg type - unnecessary coupling in my opinion.

Easier to debug: programmers are one of the users of the code, and they need to be able to easily read code as well as trace logs, audit data, whatever.


答:
Instead of having two code version with a common base you should design your application in a way to make those premium features plug-able and driven by configuration rather than different code bases.

If you are afraid to ship those premium features (disabled by configuration) with the basic version you can still remove that code in a final build/packaging step and just have two build profiles.

Having this design you can also ship 5 different flavors and get very flexible, maybe even allowing third parties to contribute.


问:
Organization of DLL linked functions
This is a code organization question.

I got my basic code working but when I expand it, it will be terrible. I have a DLL which I don't have a .lib for. Therefore I have to use the whole LoadLibrary()/GetProcAddress() combo. It works great. But this DLL that I'm referencing has 100+ functions. My current process is:

1) Typedef a type for the function. E.g.: typedef short(_stdcall *type1)(void);

2) Assign a function name that I want to use such as type1 function_1;

3) Load the DLL, then do something like: function_1 = (type1)GetProcAddress(hinstLib, "_mangled_funcName@5");

Normally I would like to do all of my function definitions in a header file but because I have to use the load library function, its not that easy. The code will be a mess. Right now I'm doing (1) and (2) in a header file and was considering making a function in another .cpp file to do the load library and dump all of the (3)'s in there. I considered using a namespace for the functions so I can use them in the main function and not have to pass over to the other function.

Any other tips on how to organize this code? My goals are to be able to use function_1 as a regular function in the main code. If I have to a ref::function_1 that would be okay but I would prefer to avoid it. This code for all practical purposes is just plain C at the moment.

答:
You can try to generate .lib file from .dll: https://wiki.videolan.org/GenerateLibFromDll/

After that the task is much simpler and cleaner - to create a header with function declarations. This can be done manually or (semi-)automated.

Title: Organization of DLL linked functions


答:
I've never done a dev test more than an hour long, and these have all been 'puzzles', a piece of work to see if I can solve problems and meet a stated aim within a given time limit.

$50 (or to me, £25-30) is a pretty poor day rate, it's like asking a plumber to fix your toilet in exchange for a drink.

My advice, in no uncertain terms, is to blog about your experience, referring to the company by name in case they're trying to create a whole app with this technique (people often google companies before heading to interview) and don't let it happen again. Next time they ask for a bug fix, you name a consultancy day-rate (at least 5 times what they've been offering) and make it known that developers will not work for free.

Being fooled is, sadly, part of life, but you don't have to sit back and accept it.


答:
I don't think they would actually use this to get cheap labor.

The reason is simple. After you write those tests, they need people to review what you write, yes reviewing code is much easier than writing the code itself, but it still takes lot of time.

And then after that they probably need people to maintain those tests, explain it, etc..

And I simply cannot imagine any IT company that would care about saving less than $100 especially companies in the US. It's never how business runs.


问:
How to name a method that both performs a task and returns a boolean as a status?
If there is a method

```<code>bool DoStuff() {
    try {
        // doing stuff...
        return true;
    }
    catch (SomeSpecificException ex) {
        return false;
    }
}
</code>```

should it rather be called IsStuffDone()?

Both names could be misinterpreted by the user:
If the name is DoStuff() why does it return a boolean?
If the name is IsStuffDone() it is not clear whether the method performs a task or only checks its result.

Is there a convention for this case? Or an alternative approach, as this one is considered flawed? For example in languages that have output parameters, like C#, a boolean status variable could be passed to the method as one and the method's return type would be void.

EDIT: In my particular problem exception handling cannot be directly delegated to the caller, because the method is a part of an interface implementation. Therefore, the caller can't be charged with dealing with all the exceptions of different implementations. It is not familiar with those exceptions. However, the caller can deal with a custom exception like StuffHasNotBeenDoneForSomeReasonException as was suggested in npinti's answer and comment.

答:
What if you simply threw the exception to the calling code?

This way you are delegating the exception handling to who ever is using your code. What if, in the future you would like to do the following:

If no exceptions are thrown, take action A

If (for instance) a FileNotFoundException is thrown, take action B

If any other exception is thrown, take action C

If you throw back your exception, the above change would simply entail the addition of an extra catch block. If you leave it as is, you would need to change the method and the place from which the method is called, which, depending on the complexity of the project, can be in multiple locations.


答:
In Java, the Collection API defines an add method that returns a boolean. It basically returns whether the add changed the collection. So for a List it will usually return true, since the item was added, while for a Set it might return false, if the item was already there, since Sets allow each unique item at most once.

That said, if you add an item that is not allowed by the collection, for example, a null value, the add will throw a NullPointerException rather than returning false.

When you apply the same logic to your case, why do you need to return a boolean? If it is to hide an exception, don't. Just throw the exception. That way you can see what went wrong. If you do need a boolean, because it might not have been done, for some other reason than an exception, name the method DoStuff(), since that represents the behavior. It does stuff.

Title: How to name a method that both performs a task and returns a boolean as a status?

Body:
If there is a method

```<code>bool DoStuff() {
    try {
        // doing stuff...
        return true;
    }
    catch (SomeSpecificException ex) {
        return false;
    }
}
</code>```

should it rather be called IsStuffDone()?

Both names could be misinterpreted by the user:
If the name is DoStuff() why does it return a boolean?
If the name is IsStuffDone() it is not clear whether the method performs a task or only checks its result.

Is there a convention for this case? Or an alternative approach, as this one is considered flawed? For example in languages that have output parameters, like C#, a boolean status variable could be passed to the method as one and the method's return type would be void.

EDIT: In my particular problem exception handling cannot be directly delegated to the caller, because the method is a part of an interface implementation. Therefore, the caller can't be charged with dealing with all the exceptions of different implementations. It is not familiar with those exceptions. However, the caller can deal with a custom exception like StuffHasNotBeenDoneForSomeReasonException as was suggested in npinti's answer and comment.


答:
In .NET, you often have pairs of methods where one of them might throw an exception (DoStuff), and the other returns a Boolean status and, on successful execution, the actual result via an out parameter (TryDoStuff).

(Microsoft calls this the "Try-Parse Pattern", since perhaps the most prominent example for it are the TryParse methods of various primitive types.)

If the Try prefix is uncommon in your language, then you probably shouldn't use it.


问:
Reporting defects in Agile

   I am working in sprint. At the end of sprint I need to send a defect report per sprint. Considering the below scenario please let me know your views.

Two teams(A & B) are working at different locations in Sprint-2 and I am a tester from Team-A and report the defects for the items developed by Team-A in each sprint

Question

1. I reported few defects in Sprint-2 for the functionality developed by Team-B in previous sprint. Do I have to consider this as observation or defect and report to Team-A?

2. I reported 5 defects of Sprint-2 for the functionality developed by team-A. All the defects are fixed and closed by me in the same sprint. Before the end of sprint I observed 2 defects got reopened for some reason. Now the defect count should be 5 or 7(5+2) should be considered for this sprint?

Thanks

Khan

答:
I think there's a fundamental issue with the definition of 'done' in your example.

If a 'defect' is identified within a sprint, I'd argue that there is no defect. The item is simply not 'done' and should be re-worked until it is. I would amend the definition of done to include the requirement that items have no known defects (if it isn't already included).

For defects identified after the sprint is finished, simply create a new item to fix the defect.

With regard to the report that you are creating, I have trouble understanding its value.

I think the place for a developer to understand that their work may be below par is either during the sprint or at the retrospective, from their peer group. Not from a report.

Also, if your definition of done is clearly stated, there is no need to let other teams know that items have defects and so, should not be considered 'done'. Items are 'done' or 'not done' at the end of a sprint, irrespective of whether defects arise later.

Title: Reporting defects in Agile


答:
When I was in a position to interview developers, those tests were short and simply "pass or fail", no bugfixing included, even when there were a few minor bugs in the code. That's because I wanted to assess the candidate's skills, not get a production ready piece of software.

The situation described in the question looks very much like someone is trying to get something useful for free (or cheap).