问:
Daily standups- yea or nay?
How valuable (or not) do you think daily stand-up meetings are?

If you're not familiar with it, this refers to a daily meeting that is part of Scrum adherents (and some other agile methodologies).  The idea is that you hold a daily meeting, timeboxed to 15 minutes, and in which everyone must stand (to encourage people to be to-the-point).

In the meeting, you go around the room and each say:
- What you did yesterday
- What you plan to do today
- Any blockers or impediments to your progress.

Do you think this practice has value?  Has anyone worked at a place that's done it, and what did you think?

答:
It can be useful, but often is not in practice.

If you have a team that does not have easy access to other team members as they work, or your organization makes it difficult to find the managers/PMs/whoever then at least you know you have one shot per day to get a question answered.

In practice it sometimes encourages people not to discuss issues immediately, and it can become a considerable time suck.  For example:

If I am only on one active development project (usually I am on two) usually there are at least one finishing out QA and one spinning up at the same time.  That is three 15 min standups a day.  Mine are nearly never back-to-back. You loose some time in making sure you are at a stopping point before each one and similarly getting back on track after each one.  Even if you assume those losses are one 10 min each that translates to more than a whole workday lost to standups each week.

Add in the commit meetings and demos and that easily eats a whole additional day.

IMHO if your team is having a communication issue then daily meetings can help, but doing them otherwise is simply too much of a resource sink.

Title: Daily standups- yea or nay?


问:
What principles does your software engineering or development organization follow?
What principles does your software engineering or development organization follow? I am very interested in seeing a list of principles from someone who works at a company where these principles are discussed, published, followed, etc.

The closest I have seen to a principles-based engineering organization are companies which are agile and follow the agile principles.

Here is a list of Marick's values/challenges: http://www.agilejourneyman.com/2010/02/4-challenges-and-5-guiding-values-of.html

I am looking for pointers to more stuff like this.  Ideally, I'd like to hear from people who have actually implemented a principles-based approach in their organization.

答:
Just found an incredible answer to this question from Tobias Mayer: http://businesscraftsmanship.wordpress.com/2010/11/15/the-artful-organization/

The Artful Organization
Posted on November 15, 2010 by tobiasmayer| 6 Comments
In today’s business climate, the focus is no longer on manufacturing “stuff” but on generating and developing knowledge. The models we have successfully used in the past to run our businesses are now shown to be outmoded and not only unhelpful but actually detrimental to success [a].  We now need to think in new ways. We need to go beyond business management, business administration and business engineering and embrace business as an art form.  It is time to usher in the artful organization [b]

The artful organization fosters a culture of openness, trust and innovation; it is a place where creativity and joy flourish as happy equals to financial success. The artful organization employs extraordinary, and sometimes counter-intuitive ideas, juxtaposed with a pragmatic approach, and interwoven with ideas of beauty, style and form to generate the tension, energy and wonderment required to reinvent the world, as we all must do.

The artful organization doesn’t require prescribed processes or new-fangled fix-all methodologies, it requires the spirit of inventiveness combined with human-centric qualities such as love, understanding and service.

I’d like to offer a set of values, principles and practices that I believe (through experience) provide a framework for change — a new approach to creating and sustaining a successful business.

5 Values

Courage — seek your edge; speak from your heart

Trust — lead from a place of faith, not suspicion; follow likewise
C* ongruency — act with integrity, so your actions and your feelings are always in alignment

Humility — acknowledge your uniqueness, celebrate your strengths, yet strive to be a worker amongst workers

Service — Be alert to the needs of others; ask for and offer help in equal proportion, for service is in the receiving as much as in the giving

7 Principles

Empiricism — experience coupled with frequent reflection allows an organization to learn fast and adapt in real time

Self-Organization — those closest to the problem have the best knowledge and intuition of how to solve the problem; create cross-functional teams to maximize the potential of this principle

Focus — a culture of abundance is created when we set priorities and minimize the amount of work in progress at any given time; completion generates happiness

Transparency — the truly open organization fosters trust and courage, and banishes fear and suspicion

Rhythm — a regular heartbeat leads to good health; flow follows cadence

Tribal Knowledge — knowledge that is kept alive through story-telling is always current, always relevant; shared ideas grow and prosper

Emergence — the best products and services are not defined, but evolve organically, adapting to their environments as they grow

5 Practices

Listen — take time to hear the needs and ideas of others; ask open, exploratory questions; practice dialog beyond discussion

Collaborate — cultivate a yes-and, because-of-that, help-me-to-see-it mindset; learn to play

Reconceive — embrace conflict, rise above compromise

Make Powerful Requests — express value, define well-formed outcomes, and create a clear separation between What and How

Be Still — maximize the amount of work not done; breathe

Applying this spiritual framework, or something like it to your business, and building processes and procedures on such a foundation will ensure you have the flexibility to stay alive and relevant, and avoid the danger of equilibrium [c].

There are no silver bullets. We all know that. There are no bullets at all, because there are no guns. We need new metaphors, and we need a new way of thinking. We have spent the last century forgetting art. Now it is time to remember — it is time to rediscover.

ContentLicense: CC BY-SA 2.5


问:
Declaration confusion in pointers
I was just confused about the following declaration in C:

```<code>char **p[5]
</code>```

I understand the char *p[] as an array of character pointers, but this one is puzzling me. Based on the precedence of [] over *, how can I interpret this?

答:
Variables in C are declared as a type followed by an identifier followed by an optional dimension.  (I'm leaving out optional modifiers, like static because they aren't germane.)  I'll start with integers since they're easy to digest:

If there's no dimension present, then the variable is of the type immediately to the identifier's left.  Declaring int x means that x is of type int.  Same with a pointer:  int *x is of type int *.

Adding a dimension (int x[5]) changes the declaration into an array, which has some side effects:

The array itself (i.e., its contents) is the variable; the identifier just becomes a way to refer to it, and in C that disqualifies it from being used as an lvalue.  This is a holdover from assembly language, where you could define a block of space in the source, give it a label and have that label's final location filled in everywhere it was used during linking.  You couldn't change the label; if you wanted a variable, it had to be in a register or memory.

The compiler will allocate enough space to hold the array (be it on the stack for an automatic, space in the object for a static or as part of a structure) and make a note of where that space is.

Use of the array's identifier as an rvalue results in it being treated as if it were a pointer to the first item* in all but two contexts (those being the & and sizeof operators).  For purposes of understanding what's been declared, treat it as a pointer: add a star to the end of the type and don't mind the dimension (e.g., int x[5] behaves like int *x when x is used as an rvalue).

Applying that to your example, we break it into its constituent parts:

```<code>char **    p    [5]
</code>```

A dimension is present, so we chop it off and add a star to the type:

```<code>char ***   p
</code>```

Used as an rvalue in all but the exceptional cases, p is the equivalent to a pointer to a pointer a pointer to char.

The problem with pointers is that they don't have any notion of where they can be pointed without exceeding array bounds.  This is another holdover from assembly, where an address might be sitting in a register with no other context and might not point at an array at all; it could just be some random instance of that type.

The sizeof operator gives you a way to determine the size of an array, because its operand is one of those exceptional cases where the identifier doesn't evaluate to a pointer:

```<code>int x[5];
size_t x_elements = sizeof x / sizeof *x
</code>```

One other note:  you ask about the relative precedences of * and [].  In a declaration, they're not operators because you're not evaluating an expression.  Ergo, precedence does not apply.

*At some point in the 30 years since I learned C, somebody started calling this decay.  As a long-time assembly programmer, I think it's a terrible term because the implication is that the by-product is somehow inferior.  There are, unfortunately, a lot of language purists who fail to understand that somebody has to write all of this low-level stuff to make their beautiful abstractions work.  K&R made that quite clear in the preface to the first edition of The C Programming Language that this is the application for which C was developed.

ContentLicense: CC BY-SA 3.0

Title: Declaration confusion in pointers

Body:
Variables in C are declared as a type followed by an identifier followed by an optional dimension.  (I'm leaving out optional modifiers, like static because they aren't germane.)  I'll start with integers since they're easy to digest:

If there's no dimension present, then the variable is of the type immediately to the identifier's left.  Declaring int x means that x is of type int.  Same with a pointer:  int *x is of type int *.

Adding a dimension (int x[5]) changes the declaration into an array, which has some side effects:

The array itself (i.e., its contents) is the variable; the identifier just becomes a way to refer to it, and in C that disqualifies it from being used as an lvalue.  This is a holdover from assembly language, where you could define a block of space in the source, give it a label and have that label's final location filled in everywhere it was used during linking.  You couldn't change the label; if you wanted a variable, it had to be in a register or memory.

The compiler will allocate enough space to hold the array (be it on the stack for an automatic, space in the object for a static or as part of a structure) and make a note of where that space is.

Use of the array's identifier as an rvalue results in it being treated as if it were a pointer to the first item* in all but two contexts (those being the & and sizeof operators).  For purposes of understanding what's been declared, treat it as a pointer: add a star to the end of the type and don't mind the dimension (e.g., int x[5] behaves like int *x when x is used as an rvalue).

Applying that to your example, we break it into its constituent parts:

char **    p    [5]

A dimension is present, so we chop it off and add a star to the type:

char ***   p

Used as an rvalue in all but the exceptional cases, p is the equivalent to a pointer to a pointer a pointer to char.

The problem with pointers is that they don't have any notion of where they can be pointed without exceeding array bounds.  This is another holdover from assembly, where an address might be sitting in a register with no other context and might not point at an array at all; it could just be some random instance of that type.

The sizeof operator gives you a way to determine the size of an array, because its operand is one of those exceptional cases where the identifier doesn't evaluate to a pointer:

int x[5];
size_t x_elements = sizeof x / sizeof *x

One other note:  you ask about the relative precedences of * and [].  In a declaration, they're not operators because you're not evaluating an expression.  Ergo, precedence does not apply.

*At some point in the 30 years since I learned C, somebody started calling this decay.  As a long-time assembly programmer, I think it's a terrible term because the implication is that the by-product is somehow inferior.  There are, unfortunately, a lot of language purists who fail to understand that somebody has to write all of this low-level stuff to make their beautiful abstractions work.  K&R made that quite clear in the preface to the first edition of The C Programming Language that this is the application for which C was developed.


问:
How is rendering a Word document different from rendering a website?
Now, it doesn't necessarily have to be Word — for ease of comparison, let's use ODT, which is based on XML — which is pretty similar to HTML. That would, to my mind, make rendering an ODT document almost like rendering an HTML website.

With ODT and HTML+CSS basically being two ways of describing a page's layout, what are the differences in rendering them?

Is it simply that HTML+CSS is more flexible and thus requires more complex rendering? A complicated website can have countless nested elements, all with relative positioning, custom styling etc. Compared to that, an ODT has a far simpler/more predictable structure, which I think should be easier to render.

答:
When your talking about rendering engines they are very different. For one thing, HTML documents have links to external resources, and are meant to present a way to navigate between pages. That's what "Hypertext" is. Word documents are meant to represent markup of a printed page. They are almost a typesetting tool.

HTML has to work and relay the information regardless of output device (screen, printer, screen printer, TTS, or others). A word document's output is either an emulated 8.5 x 11 page or a real one (or other sizes).

The very job of HTML and word documents is fundamentally different. It's basically trying to compare cars and boats. There are similarities, but there are way more differences.


答:
Let's cut to the chase, we are talking about mark-up languages and how they are displayed in a browser.

HTML is data coupled with the instructions on how to display the data. Other technologies such as CSS and Javascript, can be used to make changes to the document after it is rendered in a browser.

XML is primarily data, generally without instructions on how to display the data. XSLTs, etc can be used in conjunction with the XML to display that data in a chosen format.

ODT is XML, but by extension, as well as properties, can be transformed from text and binary resources into a graphical display, much like a HTML document is rendered in a browser.

As with anything in CS, there will be exceptions, such as an API that or some other tool that can make changes that were not envisioned by the authors of the specifications.

Browsers are designed to take HTML (text), based on the extension type, and turn that into a graphical display of those text and binary resources.

Most browsers are designed to take XML, also based on the extension type, and display a hierarchical tree of the data. That is where transformations such as XSLTs come in, they are designed to take data in a specific format, and then transform the data into something else; HTML, text, more XML, etc.

XML is primarily concerned with storing the data, by design there aren't any instructions embedded in XML that define how the data should be displayed. Custom XML schemas sometimes throw this idea right out the window and mix XML with custom elements and attributes in order to create their own markup language variant, for their own custom interpreter, such is the case with ODT and other open document types.

Since ODT is also XML, based on the extension a browser could process the data using a specific set of instructions.

Check out these links for more information on the HTML and XML specifications:

Link to HTML 5 Specification at W3C

Link to the XML Specification at W3C

Title: How is rendering a Word document different from rendering a website?

Body:
Now, it doesn't necessarily have to be Word — for ease of comparison, let's use ODT, which is based on XML — which is pretty similar to HTML. That would, to my mind, make rendering an ODT document almost like rendering an HTML website.

With ODT and HTML+CSS basically being two ways of describing a page's layout, what are the differences in rendering them?

Is it simply that HTML+CSS is more flexible and thus requires more complex rendering? A complicated website can have countless nested elements, all with relative positioning, custom styling etc. Compared to that, an ODT has a far simpler/more predictable structure, which I think should be easier to render.


问:
Why use an opaque "handle" that requires casting in a public API rather than a typesafe struct pointer?
I'm evaluating a library whose public API currently looks like this:

libengine.h

```<code>/* Handle, used for all APIs */
typedef size_t enh;

/* Create new engine instance; result returned in handle */
int en_open(int mode, enh *handle);

/* Start an engine */
int en_start(enh handle);

/* Add a new hook to the engine; hook handle returned in h2 */
int en_add_hook(enh handle, int hooknum, enh *h2);
</code>```

Note that enh is a generic handle, used as a handle to several different datatypes (engines and hooks).

Internally, most of these APIs of course cast the "handle" to an internal structure which they've malloc'd:

engine.c

```<code>struct engine
{
    // ... implementation details ...
};

int en_open(int mode, *enh handle)
{
    struct engine *en;

    en = malloc(sizeof(*en));
    if (!en)
        return -1;

    // ...initialization...

    *handle = (enh)en;
    return 0;
}

int en_start(enh handle)
{
    struct engine *en = (struct engine*)handle;

    return en->start(en);
}
</code>```

Personally, I hate hiding things behind typedefs, especially when it compromises type safety. (Given an enh, how do I know to what it's actually referring?)

So I submitted a pull request, suggesting the following API change (after modifying the entire library to conform):

libengine.h

```<code>struct engine;           /* Forward declaration */
typedef size_t hook_h;    /* Still a handle, for other reasons */

/* Create new engine instance, result returned in en */
int en_open(int mode, struct engine **en);

/* Start an engine */
int en_start(struct engine *en);

/* Add a new hook to the engine; hook handle returned in hh */
int en_add_hook(struct engine *en, int hooknum, hook_h *hh);
</code>```

Of course, this makes the internal API implementations look a lot better, eliminating casts, and maintaining type safety to/from the consumer's perspective.

libengine.c

```<code>struct engine
{
    // ... implementation details ...
};

int en_open(int mode, struct engine **en)
{
    struct engine *_e;

    _e = malloc(sizeof(*_e));
    if (!_e)
        return -1;

    // ...initialization...

    *en = _e;
    return 0;
}

int en_start(struct engine *en)
{
    return en->start(en);
}
</code>```

I prefer this for the following reasons:

Added type safety

Improved clarity of types and their purpose

Removed casts and typedefs

It follows the recommended pattern for opaque types in C

However, the owner of the project balked at the pull request (paraphrased):

Personally I don't like the idea of exposing the struct engine. I still think the current way is cleaner & more friendly.

Initially I used another data type for hook handle, but then decided to switch to use enh, so all kind of handles share the same data type to keep it simple. If this is confusing, we can certainly use another data type.

Let's see what others think about this PR.

This library is currently in a private beta stage, so there isn't much consumer code to worry about (yet). Also, I've obfuscated the names a bit.

How is an opaque handle better than a named, opaque struct?

Note: I asked this question at Code Review, where it was closed.

答:
Déjà vu

How is an opaque handle better than a named, opaque struct?

I encountered the exact same scenario, only with some subtle differences. We had, in our SDK, a lot of things like this:

```<code>typedef void* SomeHandle;
</code>```

My mere proposal was to make it match our internal types:

```<code>typedef struct SomeVertex* SomeHandle;
</code>```

To third parties using the SDK, it should make no difference whatsoever. It's an opaque type. Who cares? It has no effect on ABI* or source compatibility, and using new releases of the SDK requires the plugin to be recompiled anyway.

* Note that, as gnasher points out, there can actually be cases where the size of something like pointer to struct and void* may actually be a different size in which case it would affect ABI. Like him, I've never encountered it in practice. But from that standpoint, the second could actually improve portability in some obscure context, so that's another reason to favor the second, albeit probably moot for most people.

Third Party Bugs

Furthermore, I had even more cause than type safety for internal development/debugging. We already had a number of plugin developers who had bugs in their code because two similar handles (Panel andPanelNew, i.e.) both used a void* typedef for their handles, and they were accidentally passing in the wrong handles to the wrong places as a result of just using void* for everything. So it was actually causing bugs on the side of those using the SDK. Their bugs also cost the internal development team enormous time, since they would send bug reports complaining about bugs in our SDK, and we'd have to debug the plugin and find that it was actually caused by a bug in the plugin passing the wrong handles to the wrong places (which is easily allowed without even a warning when every handle is an alias for void* or size_t). So we were needlessly wasting our time providing a debugging service for third parties because of mistakes caused on their part by our desire for conceptual purity in hiding away all internal information, even the mere names of our internal structs.

Keeping the Typedef

The difference is that I was proposing that we do stick to the typedef still, not to have clients write struct SomeVertex which would affect source compatibility for future plugin releases. While I personally like the idea of not typedefing away struct in C, from an SDK perspective, the typedef can help since the whole point is opacity. So there I'd suggest relaxing this standard just for the publicly-exposed API. To clients using the SDK, it should not matter whether a handle is a pointer to a struct, an integer, etc. The only thing that matters to them is that two different handles do not alias the same data type so that they don't incorrectly pass in the wrong handle to the wrong place.

Type Information

Where it matters most to avoid casting is to you, the internal devs. This kind of aesthetic of hiding away all internal names from the SDK is some conceptual aesthetic that comes at the significant cost of losing all type information, and requiring us to needlessly sprinkle casts into our debuggers to get at critical information. While a C programmer should largely be accustomed to this in C, to require this needlessly is just asking for trouble.

Conceptual Ideals

In general you want to watch out for those types of developers who put some conceptual idea of purity far above all practical, daily needs. Those will drive the maintainability of your codebase to the ground in seeking some Utopian ideal, making the whole team avoid suntan lotion in a desert out of fear that it's unnatural and may cause a vitamin D deficiency while half the crew are dying from skin cancer.

User-End Preference

Even from the strict user-end standpoint of those using the API, would they prefer a buggy API or an API that works well but exposes some name they could hardly care about in exchange? Because that's the practical trade-off. Losing type information needlessly outside of a generic context is increasing the risk of bugs, and from a large-scale codebase in a team-wide setting over a number of years, Murphy's law tends to be quite applicable. If you superfluously increase the risk of bugs, chances are you'll at least get some more bugs. It doesn't take too long in a large team setting to find that every kind of human mistake imaginable will eventually go from a potential into a reality.

So perhaps that's a question to pose to users. "Would you prefer a buggier SDK or one that exposes some internal opaque names that you'll never even care about?" And if that question seems to put forth a false dichotomy, I'd say more team-wide experience in a very large-scale setting is needed to appreciate the fact that a higher risk for bugs will ultimately manifest real bugs in the long term. It matters little how much the developer is confident in avoiding bugs. In a team-wide setting, it helps more to think about the weakest links, and at least the easiest and quickest ways to prevent them from tripping.

Proposal

So I'd suggest a compromise here which will still give you the ability to retain all the debugging benefits:

```<code>typedef struct engine* enh;
</code>```

... even at the cost of typedefing away struct, will that really kill us? Probably not, so I recommend some pragmatism on your part as well, but more so to the developer who would prefer to make debugging exponentially harder by using size_t here and casting to/from integer for no good reason except to further hide information which is already 99% hidden to the user and cannot possibly do any more harm than size_t.

Body:
Déjà vu

How is an opaque handle better than a named, opaque struct?

I encountered the exact same scenario, only with some subtle differences. We had, in our SDK, a lot of things like this:

typedef void* SomeHandle;

My mere proposal was to make it match our internal types:

typedef struct SomeVertex* SomeHandle;

To third parties using the SDK, it should make no difference whatsoever. It's an opaque type. Who cares? It has no effect on ABI* or source compatibility, and using new releases of the SDK requires the plugin to be recompiled anyway.

* Note that, as gnasher points out, there can actually be cases where the size of something like pointer to struct and void* may actually be a different size in which case it would affect ABI. Like him, I've never encountered it in practice. But from that standpoint, the second could actually improve portability in some obscure context, so that's another reason to favor the second, albeit probably moot for most people.

Third Party Bugs

Furthermore, I had even more cause than type safety for internal development/debugging. We already had a number of plugin developers who had bugs in their code because two similar handles (Panel andPanelNew, i.e.) both used a void* typedef for their handles, and they were accidentally passing in the wrong handles to the wrong places as a result of just using void* for everything. So it was actually causing bugs on the side of those using the SDK. Their bugs also cost the internal development team enormous time, since they would send bug reports complaining about bugs in our SDK, and we'd have to debug the plugin and find that it was actually caused by a bug in the plugin passing the wrong handles to the wrong places (which is easily allowed without even a warning when every handle is an alias for void* or size_t). So we were needlessly wasting our time providing a debugging service for third parties because of mistakes caused on their part by our desire for conceptual purity in hiding away all internal information, even the mere names of our internal structs.

Keeping the Typedef

The difference is that I was proposing that we do stick to the typedef still, not to have clients write struct SomeVertex which would affect source compatibility for future plugin releases. While I personally like the idea of not typedefing away struct in C, from an SDK perspective, the typedef can help since the whole point is opacity. So there I'd suggest relaxing this standard just for the publicly-exposed API. To clients using the SDK, it should not matter whether a handle is a pointer to a struct, an integer, etc. The only thing that matters to them is that two different handles do not alias the same data type so that they don't incorrectly pass in the wrong handle to the wrong place.

Type Information

Where it matters most to avoid casting is to you, the internal devs. This kind of aesthetic of hiding away all internal names from the SDK is some conceptual aesthetic that comes at the significant cost of losing all type information, and requiring us to needlessly sprinkle casts into our debuggers to get at critical information. While a C programmer should largely be accustomed to this in C, to require this needlessly is just asking for trouble.

Conceptual Ideals

In general you want to watch out for those types of developers who put some conceptual idea of purity far above all practical, daily needs. Those will drive the maintainability of your codebase to the ground in seeking some Utopian ideal, making the whole team avoid suntan lotion in a desert out of fear that it's unnatural and may cause a vitamin D deficiency while half the crew are dying from skin cancer.

User-End Preference

Even from the strict user-end standpoint of those using the API, would they prefer a buggy API or an API that works well but exposes some name they could hardly care about in exchange? Because that's the practical trade-off. Losing type information needlessly outside of a generic context is increasing the risk of bugs, and from a large-scale codebase in a team-wide setting over a number of years, Murphy's law tends to be quite applicable. If you superfluously increase the risk of bugs, chances are you'll at least get some more bugs. It doesn't take too long in a large team setting to find that every kind of human mistake imaginable will eventually go from a potential into a reality.

So perhaps that's a question to pose to users. "Would you prefer a buggier SDK or one that exposes some internal opaque names that you'll never even care about?" And if that question seems to put forth a false dichotomy, I'd say more team-wide experience in a very large-scale setting is needed to appreciate the fact that a higher risk for bugs will ultimately manifest real bugs in the long term. It matters little how much the developer is confident in avoiding bugs. In a team-wide setting, it helps more to think about the weakest links, and at least the easiest and quickest ways to prevent them from tripping.

Proposal

So I'd suggest a compromise here which will still give you the ability to retain all the debugging benefits:

typedef struct engine* enh;

... even at the cost of typedefing away struct, will that really kill us? Probably not, so I recommend some pragmatism on your part as well, but more so to the developer who would prefer to make debugging exponentially harder by using size_t here and casting to/from integer for no good reason except to further hide information which is already 99% hidden to the user and cannot possibly do any more harm than size_t.


问:
Why SQL numeric precision is specified as the number of digits?
I understand that this might be a loaded question.
For example, in Sql Server 2008, doing DECLARE @someNum NUMERIC(10); means that the range of the variable will be limited by 10 digits. In other languages such as C, Java, etc. the standards for representing whole and fractional numbers are different. The range of the values is being sliced by multiples of bytes, and this is understandable, due to the way hardware works. A language could provide something that most hardware does not yet support, such as a 256-bit integer type, which will work fast on a 64-bit computer, if it is implemented correctly.

The numeric types in the databases seem to be not tied to underlying hardware. I understand that the values might need to be null-ablle, which can cost a bit or two, but why such arbitrary precision that does not seem to optimize storage or calculation speed?

Perhaps databases do not need to perform numeric computations often. Perhaps digit width of a column is a natural extension of character width. Perhaps SQL was supposed to be a non-programmers language (because it sounds like English :) )?
Perhaps history took a weird turn, such as one that allowed the inferior VHS format to prosper.

Please help me understand this.

答:
In other languages such as C, Java, etc. the standards for representing whole and fractional numbers are different.

In C and various programming languages, integers (INT() in SQL) are usually "packed" integer bit strings and floating point numbers (FLOAT()/DOUBLE() in SQL) are usually stored as IEEE-754/-854-compliant bit strings.

DECIMAL() and NUMERIC() typically use Binary Coded Decimal (BCD).

A packed, signed, 32-bit integer will store -2-billion-or-so to +2-billion-or-so.  You can reliably use 9 digits, +/-, but not 10.  For more than 10, you need the next step up in integer, which will likely be 64-bits.

IEEE-754 deals with storing floating point numbers in a 32-bit sequence.  It stores:

one bit for the sign

multiple bits for a relative exponent

multiple bits for a mantissa

In this fashion, it holds several significant digits and a range of exponents.  The order of the bits is such that you can treat them as 32-bit signed integer, for comparison purposes, and you should get the right answer.  This way, you don't have to have different ways for comparing integers vs floats.  IEEE-854 takes this to 64-bits, allocating more bits to the relative exponent and the mantissa, allowing more digits of precision and a greater range of exponents.

The problem with these floating point values is that they are ALL approximations.  And they (at least, the older, more commonly supported formats) are all based on exponents of 2, not exponents of 10.  Ergo, there is no EXACT representation, in either one, for 1.1 or 1.01; there's no exponent of 2 which you can multiply either of those by and get an exact value.  If you deal with financial transactions, you don't want an approximation.  If you must round, you want to deal with exponents of 10, not 2, because that's how we denominate money.

DECIMAL() and NUMERIC() are different animals, using BCD.  If you define a DECIMAL(7,2), that will store a total of 7 decimal digits, keeping track of the fact that 2 of them are behind the decimal point.  That's 7 significant digits, no approximation.  That will likely fit in 32-bits of space (4 bits for each digit + 4 bits for the total field, to store sign, null, etc.).  If you multiply two DECIMAL(7,2) fields, you know PRECISELY, down to the penny, what you will get.  No approximations.

And the aforementioned DECIMAL(10) will likely only need 48 bits of space to store.  And you can use all 10 digits, positive and negative.

Perhaps databases do not need to perform numeric computations often. Perhaps digit width of a column is a natural extension of character width. Perhaps SQL was supposed to be a non-programmers language (because it sounds like English :) )? Perhaps history took a weird turn, such as one that allowed the inferior VHS format to prosper.

Afraid I have to disagree with you, there.  The databases I deal with do a LOT of numeric calculations.  Most of it, though, is dollars and cents, which largely REQUIRES the precision of BCD and the associated DECIMAL(x,y) formats.  The fact that BCD chews up 4 bits / digit, while ASCII used 8, is mostly coincidental.

The invention of SQL is credited to E. F. Codd, who was working at IBM.  No surprise, therefore, that the formatting for DECIMAL(x,y) is largely stolen from FORTRAN.


问:
What is "Split Data Model", mentioned in the book "Java Concurrency In Practice"?
I'm learning Java multithreading programming by the book "Java Concurrency In Practice". In chapter 9.4.2 Split Data Model, I read this:

From the perspective of the GUI, the Swing table model classes like TableModel and treeModel are the official
  repository for data to be displayed. However, these model objects are often themselves "views" of other objects
  managed by the application. A program that has both a presentation‐domain and an application domain data model is
  said to have a split‐model design (Fowler, 2005).

In a split‐model design, the presentation model is confined to the event thread and the other model, the shared model,
  is thread‐safe and may be accessed by both the event thread and application threads. The presentation model registers
  listeners with the shared model so it can be notified of updates. The presentation model can then be updated from the
  shared model by embedding a snapshot of the relevant state in the update message or by having the presentation
  model retrieve the data directly from the shared model when it receives an update event.

The snapshot approach is simple, but has limitations. It works well when the data model is small, updates are not too
  frequent, and the structure of the two models is similar. If the data model is large or updates are very frequent, or if one
  or both sides of the split contain information that is not visible to the other side, it can be more efficient to send
  incremental updates instead of entire snapshots. This approach has the effect of serializing updates on the shared
  model and recreating them in the event thread against the presentation model. Another advantage of incremental
  updates is that finer‐grained information about what changed can improve the perceived quality of the display if only
  one vehicle moves, we don't have to repaint the entire display, just the affected regions.

What is the meaning of "Split Data Model"? Can you show me an example to help me understand it?

答:
The presentation domain data model is concerned with rendering the application domain data model on the screen.  For example, let's say you have a table in your application data model, and want to present that data, filtered and/or sorted as the user chooses.  In that case, the choice of filter and/or sort is an aspect of the presentation domain.  (The filter/sort can be represented as data and persisted, even, should the user want to retrieve that same view.)

By separating the concerns of presentation model from application model, it is easy for us to create an application that can have multiple simultaneous views (presentations) of the same application data.  Imagine one user having multiple windows open looking at the same application data, though probably presented somewhat differently (in our example, maybe sorted or filtered differently).

Now, also imagine two or more users looking at the same data (possibly with different filter/sort).  Once we have multiple users potentially altering the same data (via their individual presentations), we need to apply multi-threaded coordination (transactions, locks, or other) over the application model.

By contrast, the presentation model won't need threading coordination (unless you persist views and allow multiple users to edit them).

OwnerDisplayName: FireSun


问:
Client Side Filtering or Server Side Filtering
So I am building a web-app. The app will be hosted on heroku and I using a MEAN Stack for development. The main purpose of the app is to allow users to search through data and be able to find a document they are looking for.

This is for my companies internal documents as most of our employees are all over the world and they need a way to find data easily.

The Idea:

The idea I have come up with is to create a web-app which provides them with a interface that allows them to search and filter data.

The filtering options provided on the web-app will be similar to that of eBay (see below)

The Data:

Initially the data set will be low. But with time it will grow quite big and I want it to be scalable to it can be used for a long time and not break or slow down as data increases.

Just a note that the data will mostly only be text. Any files such as pdf, excel or another formats will be saved on external resources like a central dropbox account and then the links for those files will be added to the web app.

The Question:

To provide the user the option to filter the data whats the best way?
When the user fill out the filtration form as the image attached above should the filtration be done on Server side and then sent to the client or should it be performed client side?

In my opinion server side is the best way to go because I can keep the whole logic of the system on the server and keep the client side code clean.

Also note that initially this will be a web-app but in future we will create a iPhone app for this as well.

Thanks in advance.

答:
If the dataset was expected to stay small, it would probably be faster to filter client-side. But because of this:

Initially the data set will be low. But with time it will grow quite
  big and I want it to be scalable to it can be used for a long time and
  not break or slow down as data increases.

I would recommend server-side filtering. Otherwise, you will have to send the client the whole dataset in order for the client to filter it.


问:
Improving sampling algorithm
I am having a bit of trouble designing a new feature at the moment. It is part of a resource management system. I was wondering if anyone has experience doing anything similar.

I'll try to explain:

Resource: a Person, Place, or Thing

Availability Template (AT): allows you to define a standard availability pattern for a type of resource. e.g.: Mon-Fri 9-5.

Resource Additional Availability (AA): allows you to define one-off availability for a resource. e.g.: Bob's overtime 10-3 on Saturdays.

Resource Availability Exclusions (AE): allows you to define when a resource is not available. e.g: Room 4 is cleaned between 4-5 on Fridays.

It's easy enough to check whether or not a resource is available at a point in time: Availability = (AT(rID, time) || AA(rID, time)) && !AE(rID, time).

But I need to be able to query a resource's availability over a time period. i.e.: "Is Room 4 available at 9am for 2 hours on Thursday?".

So far, I have created an algorithm that "samples" the availability at a specific interval over the time period. However, this means that the returned availability lags the actual availability and some information might be missed.

E.g.: if I sample the availability every 15 minutes between 9 and 11, it would miss an exclusion between 9:05 and 9:10.

I could use a very small sampling period (eg: 1 minute) but that might not be performant and overall it is still quite an ugly brute-force approach.

Are there any standard patterns or algorithms for this type of problem?

答:
First, reduce the problem to a test of availabilities "per day". All your examples are using ranges for only one day, but if you also want to support a range like "Monday 10:00 to Wednesday 14:00", break this down into three tests for "Monday 10:00 to 23:59", "Thursday 00:00 to 23:59" and "Wednesday 00:00 to 10:00".

For each day, store an interval representation of the available times in form of a list of intervals. Initially, the list contains only one interval, lets say [9:00 , 17:00] for each day, and whenever a part of the day gets blocked for an different interval, (for example, room gets blocked from 10:00 to 11:25), you calculate the new resulting list of intervals (here [9:00,10:00], [11:25, 17:00]). Additional availability leads to additional intervals for some dates, and the "availability check" is nothing but an overlapping test of one new interval with all "available time" intervals of a specific day.

So in terms of sets, all you need is a "subtraction" operation on intervals, a "union" operation, and an overlapping test.

The implementation of the overlapping test for two intervals is not hard, it does not require any "sampling", just comparing the minimum and maximum values of each interval. See here for an example. Creating interval unions, or "subtraction" of intervals is not too complicated, too, see here, note that the "set difference" of two intervals yields either two new intervals, one new interval or zero intervals . At the second link, you will also find a reference to an interval package in Python. I guess there are similar components or code snippets available for each major programming language, if you don't want to implement this on your own.

Title: Improving sampling algorithm


问:
Is it effective to review code in language I don't know?
I'm an experienced developer, but have not done many code reviews. I'm being asked to review code written in Python but I do not know Python.

Does it make any sense at all to review code in a language I don't know?

答:
That depends on what the goal of the review is; i.e. what you mean by effective.

You will still likely be able to detect some problems. If you're all they got to review with and they're just hoping that you taking a look over it helps some and possibly catches something, then sure. Many concepts of structure are similar between languages. One especially is being able to review the commenting. It should be commented well enough that a programmer not of that particular language should still be able to get a good feel of what's happening. If not...then you can tell them where their commenting is lacking. If it is that well commented...then you should be able to review quite a bit of their structure just through the annotations of what's going on rather than actually reading the code of what's going on.

But you will not likely detect many other problems. So if they intend for your review to be an exhaustive determination of whether or not this is a well made / workable program they will be disappointed.

Whether or not that outcome is worth the time of you doing it depends largely on the project.


问:
Design choices when doing multithreading in C++
OK, so, background:

I have a little "server" app/module that is written in C++. This is kindof a tunnel application that provides a standardized API via TCP/IP[a] and on the other hand talks to the "real" server via TCP/IP[a] via it's own interface.

It doesn't even officially support multiple clients, but due to the inherently multithreaded nature of the processing that is done by the server and by the client (plus asynchronous callbacks) the tunneling app itself is multithreaded and runs pretty well during normal operations.

The application uses a number of simple mutexes (Win32 CRITICAL_SECTIONS) to lock down the code paths that would create problems when executed concurrently and this does work pretty well in practice.

A little picture:

```<code>[proprietary server]        [tunneling app]        [client]
                     <--1-                  <--1-
                              (upcalls)    |<--2-
                     <--3-                  <--3-
                                           | -2-->
                      --1->    (returns)     -1-->
                      --3->                  -3-->
                      -a-->    (callbacks)   -a1->
                      -b--> |                -a2->
</code>```

So pretty much everything goes with regards to call chains.

We have now identified a number of problems in fringe cases like calls during initialization or shutdown of the server and/or tunneling app and I'm struggling to properly fix these as they are mostly related to multithreaded object and thread lifetimes and just throwing more mutexes or more locks at the problem really doesn't cut it.

As for the Question:

What I am looking for is advice and guideline on how to write multithreaded (C++) code that would work in all corner cases from the start and doesn't involve just locking everything down. Possibly with a specific focus on the context of heavy multithreading in a network communication scenario.

Any insights welcome.

Notes:

[a] : It's a CORBA communication interface. But The problems I see are really not related to the actual messaging mechanism used, as the messaging part on the wire works very well.

答:
Micro-Facets of Functionality

This is going to be more of a general design strategy answer, without understanding the precise issues you are dealing with in a networking context. It might completely miss. Yet understanding this was so helpful to me in just software engineering in general that I'm eager to share it, and I think it has a good chance of helping. Simplifying state/resource management and control flows tends to go hand-in-hand with simplifying multithreading, which is handy.

To me the perceived sense of overwhelming complexity of a very large-scale codebase is often owed to the fact that it's structured in a way that has us zooming in to a micro-level of teeny islands of functionality and control flows and then trying to have us reason about how it functions abroad at an overseer level. In that case, it's making us try to sum up little pieces of functionality to try to reason about the codebase, and it's hard to try to work bottom-up in this fashion and piece together what's actually going on.

This problem is multiplied when you have a central data structure, with all these little teeny micro-facets of functionality all weaving and winding and forming a maze of functionality which ultimately works their way towards calling functions which update this central data structure.

Such a codebase can be perfectly SOLID in its engineering. All entities might perform a singular responsibility, all dependencies flow towards abstractions, everything is easily extensible that needs to be extensible, etc.

Regardless, the conceptual complexity of the codebase seems beastly when trying to comprehend it, and trying to achieve both thread safety and thread efficiency might seem like a hopeless endeavor. It's because this twisting maze of control flow still makes it very difficult to reason about when and where critical things happen. Such systems make us have to zoom in, making us trying to understand the big picture from little puzzle pieces.

"When does this shared, central data structure get updated?"

"Well, it might be updated when this event is triggered when the
user pushes this button which causes this function to get called which
causes that function to get called which updates this structure. It
might also get updated when this event is triggered which calls
functions A->B->C->D->E....->Z which then updates this structure, etc. etc. etc."

It's hard to reason about when and how this shared state is accessed and modified, and when it's hard to reason about that, the temptation is often to lock with every little hand that touches it.

Flat, Bulk, Deferred Processing (Mass Consumer Design)

The solution to me here to both alleviate perceived complexity and also kind of flatten the whole system and make it easier to multithread efficiently with smarter locks is to design the system in a way that allows you to zoom out. Instead of this:

We do something like this:

By doing this, you centralize one single place in the entire system that can actually modify this data structure. And that one place can potentially multithread to its hearts' content, using one of the simplest means of multithreading -- the parallel for loop. Such a place then can process many requests at once, and thus lock at a more appropriately coarse/granular level. It'll also make it easier to split up the central data structure, for example, and reduce the contention that way with only the central processor to update in response.

Just about everything is easier when you avoid having a teeny little isolated pipeline for each little request that can occur, and instead have a bulk, centralized pipeline that handles multiple requests at once. Just at a meta level, reasoning about such a codebase's behavior starts to become like reasoning about a dozen continents rather than thousands of little islands.

In such a case, each micro-facet (each client, e.g.) might push or write some local state which is then read/popped/cleared by this deferred, flat, bulk processor. A general approach is have each one push to a local, concurrent queue (producer) that gets popped by the bulk processor (mass consumer), though a queue is overkill if you can just have some atomic variables you can read/write.

We have now identified a number of problems in fringe cases like calls
during initialization or shutdown of the server and/or tunneling app
and I'm struggling to properly fix these as they are mostly related to
multithreaded object and thread lifetimes and just throwing more
mutexes or more locks at the problem really doesn't cut it.

By having one centralized place which does all the heavy-lifting, it's easier to multithread but also avoid issues like these associated with attempts to update shared state once destroyed or prior to initialization. This kind of awkwardness with initialization and destruction order can also be understood and tackled from untangling state management and associated control flow. The central, mass consumer (bulk processor) would simply ignore the local state of clients (not consume and therefore not do anything with shared state) when the system is not ready (pre-startup or post-shutdown). The clients are still free to safely push/write local state, since they are not modifying any shared/central state in doing so. The centralized system will pull/process that state when it's actually ready to do so.

If this kind of asynchronous, deferred processing is applicable, it can tremendously simplify all kinds of goals ranging from correctness to simplicity to safety and efficiency. Most of all, codebases structured like this just tend to be easier to comprehend for humans at a broad level without thinking so much at a micro-facet level, the "what if?" scenarios become few in number, and life becomes a lot simpler.

If there are multiple phases involved (like a state machine), then you can have more than one centralized bulk processor. The first could push results back to each entity ("micro-facet"). The second system then consumes those results and processes the second phase in a serial fashion for order-specific, multi-phase processing.

Optimization

To avoid needlessly looping through micro-facets which don't need to do anything, one strategy is to aggregate them into blocky data structures, like so:

This aggregates 4 "micro-facet" entities into a each unrolled block (kind of like an unrolled linked list but without shuffling contents around to keep nodes at least half full). You can keep track of vacant spaces in the list and reclaim them upon subsequent insertions. And of course you can aggregate more than 4 of these, maybe even like 64 at once, and can even form a hierarchy (I just used 4 because it's easy to draw).

The key idea is that you can skip N (4 in this diagram) micro-facets at once to process if this Needs Processing atomic flag is not set, so it's kind of like a miniature skip list (and can be expanded to a full-blown hierarchy for maximum scalability). Any time one of these entities have work that needs to be done, they can set the Needs Processing flag in the block they reside in (and possibly upwards to the root of the hierarchy if a full-blown hierarchy is used). We can also use condition variables to put this deferred processing thread to sleep when there's nothing to do.

The bulk processor works its way downwards from these blocks that have Needs Processing set to true down to the individual entities that need to be processed within that block.

CES/ECS

This is similar to the design of entity-component systems used in games, but playing especially to the "systems" part. Systems as bulk processors can really help reason about the software, as well as multithread more efficiently when each system is doing a lot of work. Even games spanning millions of lines of code might only have a couple dozen systems which are the sole places doing the critical work, and that's really easy to digest for the human brain when there's only a couple of dozens of places in the codebase that are responsible for the most critical updates to the software's core state, instead of tens of thousands of miniature functions to think about.

The key is not to confine yourself into a little tiny island of functionality, and to avoid all such little teeny islands communicating directly with central data structures. Make it one big island of bulk processing that has sole access to the central data, make each little island which is responsible only for its local state, and life will become a lot easier.

The control flow will become untangled in the places where it's most critical, and likewise the temptation to lock on every little teeny request will also go away. Your call stacks will also become very shallow. Instead we have end up having big, chunky bulk work to do in a handful of centralized places, and that leaves a lot of breathing room to lock at more appropriate levels, and less frequently with less contention. Most importantly, it should have you dealing with bugs a lot less when the question of "when and where" the critical things happen, like updating central data, is no longer such a mystery that requires reaching for a magnifying glass and sifting through the entire codebase.


问:
Representing a number in 12 Bit 2's Complement Fixed Point
The original question is as follows:

Consider a 12-bit two's complement fixed point representation, with 8 bits for the integer part and 4 bits for the fraction.

  What is the binary encoding of the decimal value 45.625?

I found the binary representation for 4510 in 8 bits to be 001011012

I then split the .62510 into a .510 and a .12510

.510 in binary is .12

.12510 in binary is .0012

Therefore, I came up with the final answer of 00101101.10102

However, this was counted incorrect.  Where did I go wrong?

答:
Answer to your question:

If there is any reason why it was counted incorrect, it would be in the notation you used.  I don't know what the expected notation was, but binary numbers typically don't have decimal points, so for me the correct answer would simply be 0010 1101 1010 or 001011011010 or even 0b001011011010.  I would hope that your professor/teacher would make format requirements clear.

Verifying the answer you received:

The answer you came up with is definitely correct, but I want to give a different approach for achieving the same result.  Normally when we convert from binary to decimal we think of powers of two.  eg:

```<code>(Bit #) (Decimal value)
...       ...
4         16
3         8
2         4
1         2
0         1
</code>```

But consider the following 'mapping' between bits and decimal numbers:

```<code>11   128 (Actually becomes the sign bit in 2s complement)
10   64
9    32
8    16
7    8
6    4
5    2
4    1
.    (just for reference where you're putting the decimal point)
3    0.5
2    0.25
1    0.125 (provided in the example given)
0    0.0625
</code>```

In your problem statement you say .0012 == .12510.  From that, we can actually build the above table. What we're trying to do is come up with a scaling factor between the two tables above.  How do we do that? Look at bits in the same position, and make a ratio between the two.  In the top table, bit 0 equals 1.  In the bottom table, bit 0 equals 0.0625.  We choose bit 0 because on one side it's equal to 1 which makes calculations easier.  These don't really have units, but if they did it might be something like:

```<code>1 standard binary encoding (sbe) = 0.0625 our encoding (oe)
</code>```

Using our resolution, we can essentially do dimensional analysis on the 'units'. Then we can crunch the calculation (units cancel and we're left with 'sbe'):

```<code>45.625 oe * (1 sbe / 0.0625 oe) = 730 sbe
</code>```

And if you look at the 'standard' binary value of 730, you'll see: 0010 1101 1010 or as you put it in your question: 00101101.1010.

Body:
Answer to your question:

If there is any reason why it was counted incorrect, it would be in the notation you used.  I don't know what the expected notation was, but binary numbers typically don't have decimal points, so for me the correct answer would simply be 0010 1101 1010 or 001011011010 or even 0b001011011010.  I would hope that your professor/teacher would make format requirements clear.

Verifying the answer you received:

The answer you came up with is definitely correct, but I want to give a different approach for achieving the same result.  Normally when we convert from binary to decimal we think of powers of two.  eg:

(Bit #) (Decimal value)
...       ...
4         16
3         8
2         4
1         2
0         1

But consider the following 'mapping' between bits and decimal numbers:

11   128 (Actually becomes the sign bit in 2s complement)
10   64
9    32
8    16
7    8
6    4
5    2
4    1
.    (just for reference where you're putting the decimal point)
3    0.5
2    0.25
1    0.125 (provided in the example given)
0    0.0625

In your problem statement you say .0012 == .12510.  From that, we can actually build the above table. What we're trying to do is come up with a scaling factor between the two tables above.  How do we do that? Look at bits in the same position, and make a ratio between the two.  In the top table, bit 0 equals 1.  In the bottom table, bit 0 equals 0.0625.  We choose bit 0 because on one side it's equal to 1 which makes calculations easier.  These don't really have units, but if they did it might be something like:

1 standard binary encoding (sbe) = 0.0625 our encoding (oe)

Using our resolution, we can essentially do dimensional analysis on the 'units'. Then we can crunch the calculation (units cancel and we're left with 'sbe'):

45.625 oe * (1 sbe / 0.0625 oe) = 730 sbe

And if you look at the 'standard' binary value of 730, you'll see: 0010 1101 1010 or as you put it in your question: 00101101.1010.


答:
HTML documents have pre-defined tags, whilst XML does not.  Because the tags are defined, browsers can be made to render the display.

A  tag has a specific meaning in HTML and is treated as such.

Now consider this XML fragment:

```<code><BODY>
 <ARM></ARM>
 <EYECOLOR></EYECOLOR>
</BODY>
</code>```

The  tag has a different context and thus is treated differently than an HTML tag.

As indicated by @MichealIT, ODT also has specific defined format for it's tags, so any software or browser will need to adhere to that definition in order to render the document.  There are probably many differences between an HTML document versus an ODT one.

Body:
HTML documents have pre-defined tags, whilst XML does not.  Because the tags are defined, browsers can be made to render the display.

A <BODY> tag has a specific meaning in HTML and is treated as such.

Now consider this XML fragment:

<BODY>
 <ARM></ARM>
 <EYECOLOR></EYECOLOR>
</BODY>

The <BODY> tag has a different context and thus is treated differently than an HTML tag.

As indicated by @MichealIT, ODT also has specific defined format for it's tags, so any software or browser will need to adhere to that definition in order to render the document.  There are probably many differences between an HTML document versus an ODT one.


问:
Is there a standard way or standard alternative to packing a struct in c?
When programming in C I have found it invaluable to pack structs using GCCs __attribute__((__packed__)) attribute so I can easily convert a structured chunk of volatile memory to an array of bytes to be transmitted over a bus, saved to storage or applied to a block of registers. Packed structs guarantee that when treated as an array of bytes it will not contain any padding, which is both wasteful, a possible security risk and possibly incompatible when with interfacing hardware.

Is there no standard for packing structs that works in all C compilers? If not then I am an outlier in thinking this is a critical feature for systems programming? Did early users of the C language not find a need for packing structs or is there some kind of alternative?

答:
When programming in C I have found it invaluable to pack structs using
  GCCs __attribute__((__packed__)) [...]

Since you mention __attribute__((__packed__)), I assume your intention is to eliminate all padding within a struct (make each member have a 1-byte alignment).

Is there no standard for packing structs that works in all C
  compilers?

... and the answer is "no". Padding and data alignment relative to a struct (and contiguous arrays of structs in stack or heap) exist for an important reason. On many machines, unaligned memory access can lead to a potentially significant performance penalty (though becoming less on some newer hardware). In some rare-case scenarios, misaligned memory access leads to a bus error that is unrecoverable (may even crash the entire operating system).

Since the C standard is focused on portability, it makes little sense to have a standard way to eliminate all padding in a structure and just allow arbitrary fields to be misaligned, since to do so would potentially risk making C code non-portable.

The safest and most portable way to output such data to an external source in a way that eliminates all padding is to serialize to/from byte streams instead of just trying to send over the raw memory contents of your structs. That also prevents your program from suffering performance penalties outside of this serialization context, and will also allow you to freely add new fields to a struct without throwing off and glitching the entire software. It'll also give you some room to tackle endianness and things like that if that ever becomes a concern.

There is one way to eliminate all padding without reaching for compiler-specific directives, though it's only applicable if the relative order between fields does not matter. Given something like this:

```<code>struct Foo
{
    double x;  // assume 8-byte alignment
    char y;    // assume 1-byte alignment
               // 7 bytes of padding for first field
};
</code>```

... we need the padding for aligned memory access relative to the address of the structure containing these fields, like so:

```<code>0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
x_______y.......x_______y.......x_______y.......x_______y.......
</code>```

... where . indicates padding. Every x must align to an 8-byte boundary for performance (and sometimes even correct behavior).

You can eliminate the padding in a portable way by using an SoA (structure of array) representation like so (let's assume we need 8 Foo instances):

```<code>struct Foos
{
   double x[8];
   char y[8];
};
</code>```

We've effectively demolished the structure. In this case, the memory representation becomes like this:

```<code>0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
x_______x_______x_______x_______x_______x_______x_______x_______
</code>```

... and this:

```<code>01234567
yyyyyyyy
</code>```

... no more padding overhead, and without involving misaligned memory access since we're no longer accessing these data fields as an offset of a structure address, but instead as an offset of a base address for what is effectively an array.

This also carries the bonus of being faster for sequential access as a result of both less data to consume (no more irrelevant padding in the mix to slow down the machine's relevant data consumption rate) and also a potential for the compiler to vectorize processing very trivially.

The downside is that it's a PITA to code. It's also potentially less efficient for random access with the larger stride in between fields, where often AoS or AoSoA reps will do better. But that's one standard way to eliminate padding and pack things as tightly as possible without screwing with the alignment of everything.

Body:
When programming in C I have found it invaluable to pack structs using
  GCCs __attribute__((__packed__)) [...]

Since you mention __attribute__((__packed__)), I assume your intention is to eliminate all padding within a struct (make each member have a 1-byte alignment).

Is there no standard for packing structs that works in all C
  compilers?

... and the answer is "no". Padding and data alignment relative to a struct (and contiguous arrays of structs in stack or heap) exist for an important reason. On many machines, unaligned memory access can lead to a potentially significant performance penalty (though becoming less on some newer hardware). In some rare-case scenarios, misaligned memory access leads to a bus error that is unrecoverable (may even crash the entire operating system).

Since the C standard is focused on portability, it makes little sense to have a standard way to eliminate all padding in a structure and just allow arbitrary fields to be misaligned, since to do so would potentially risk making C code non-portable.

The safest and most portable way to output such data to an external source in a way that eliminates all padding is to serialize to/from byte streams instead of just trying to send over the raw memory contents of your structs. That also prevents your program from suffering performance penalties outside of this serialization context, and will also allow you to freely add new fields to a struct without throwing off and glitching the entire software. It'll also give you some room to tackle endianness and things like that if that ever becomes a concern.

There is one way to eliminate all padding without reaching for compiler-specific directives, though it's only applicable if the relative order between fields does not matter. Given something like this:

struct Foo
{
    double x;  // assume 8-byte alignment
    char y;    // assume 1-byte alignment
               // 7 bytes of padding for first field
};

... we need the padding for aligned memory access relative to the address of the structure containing these fields, like so:

0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
x_______y.......x_______y.......x_______y.......x_______y.......

... where . indicates padding. Every x must align to an 8-byte boundary for performance (and sometimes even correct behavior).

You can eliminate the padding in a portable way by using an SoA (structure of array) representation like so (let's assume we need 8 Foo instances):

struct Foos
{
   double x[8];
   char y[8];
};

We've effectively demolished the structure. In this case, the memory representation becomes like this:

0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
x_______x_______x_______x_______x_______x_______x_______x_______

... and this:

01234567
yyyyyyyy

... no more padding overhead, and without involving misaligned memory access since we're no longer accessing these data fields as an offset of a structure address, but instead as an offset of a base address for what is effectively an array.

This also carries the bonus of being faster for sequential access as a result of both less data to consume (no more irrelevant padding in the mix to slow down the machine's relevant data consumption rate) and also a potential for the compiler to vectorize processing very trivially.

The downside is that it's a PITA to code. It's also potentially less efficient for random access with the larger stride in between fields, where often AoS or AoSoA reps will do better. But that's one standard way to eliminate padding and pack things as tightly as possible without screwing with the alignment of everything.


问:
Why would I need `equals` if I have already `hashcode`?
I got a question about why we need equals if we have hashcode.

My first attempt was the answer because collision. But we corrected starting point with the assumption that we have not many objects so there is no collision at all.

My second attempt was the answer because of the speed. But I also got the reply that there is something conceptual difference between hashcode and equals.

So I read a lot of posts, the java doc and can not find the answer. Do I miss something?

答:
To put it bluntly: they serve different purposes.

Equals is for testing equality.

Hashcode is in order to produce an int (hopefully well distributed)

if your hashcode is unique, you could as well use it for equality. However, for many kind of objects, producing such a unique hash is plainly impossible.

Imagine an object containing two ints a and b. There is no way to produce a unique int hash for such an object. So you still need equality to compare them.

As for the comparator, it's usually a bad idea to use the output as hash because the result is not well distributed, resulting in lots of collisions when used in hash maps or similar data structures.


答:
Let's say you have a 64 bit hash code. There are 2^64 possible hash codes. Now lets say you are hashing strings. There are many, many more than 2^64 strings. There are 2^64 strings of eight 8-bit characters alone. There are 256 times more strings of nine 8-bit characters and even more strings of 10, 11 or more charcters. Obviously there will be many, many 9 character and even more longer strings with the same hash code. </p>

<p>Therefore you can't rely on the hash code alone. Equal hash codes don't guarantee that the objects are identical. </p>

Title: Why would I need `equals` if I have already `hashcode`?

Body:
I got a question about why we need equals if we have hashcode.

My first attempt was the answer because collision. But we corrected starting point with the assumption that we have not many objects so there is no collision at all.

My second attempt was the answer because of the speed. But I also got the reply that there is something conceptual difference between hashcode and equals.

So I read a lot of posts, the java doc and can not find the answer. Do I miss something?

Body:
Let's say you have a 64 bit hash code. There are 2^64 possible hash codes. Now lets say you are hashing strings. There are many, many more than 2^64 strings. There are 2^64 strings of eight 8-bit characters alone. There are 256 times more strings of nine 8-bit characters and even more strings of 10, 11 or more charcters. Obviously there will be many, many 9 character and even more longer strings with the same hash code.

Therefore you can't rely on the hash code alone. Equal hash codes don't guarantee that the objects are identical.


问:
Methods of debugging code (Nightmare situation)
I am frequently tasked with debugging an application at my job. It is a BI Application that we deploy to businesses, which includes a test environment, and a production environment. I am wondering if there are any apps/tools/methods that people can suggest, based on these constraints:

Debugger cannot be used on client site or locally, because the software depends on custom third party applications that we don't have test environments for. (EDIT: to be fair, it is possible to debug locally in some cases. If we use nothing but the core code. Much of the problematic code resides in a dll that encapsulates third party specific communication: sockets, process pipes, soap calls, custom logic that changes the behavior of core code. Typically during an implementation or enhancement for a client, we would be writing new code to this area.)

There is virtually no logging done in our apps. There are no unit tests.

Version control only has 1 version of the full solution (using source safe 2005). So it's not possible to get a previous version of the entire solution, only individual files. (Unless someone knows ways around this).

Cannot reproduce locally, often times cannot reproduce on test environment (High chance that test and production are not the same version).

There is a high chance that the version the client is using is different from the one on source safe. This is because individual files are updated, which have embedded custom logic for that specific client. Often what happens is an update is made to a binary, which requires changes to several other binaries, but when a commit is done, no one has any record or knowledge of this. A somewhat common error I see is the 'Function/Method not found' or 'Method call has too many/too few parameters specified' on a clients environment.

This is a .net VB solution

Cannot install any software on client sites, but can locally

Our application is extremely customizable, but unfortunately the customization logic is spread out across all the classes and files, from the front end all the way to the data layer, including custom changes made to the database on a per client basis.

There are virtually no comments in the code. There is no documentation about the architecture. No documentation about the api. The only thing we have are hundreds upon hundreds of email chains that somewhat explain what's going on. The only people that know the code are the ones that originally wrote it, but they're no longer developers per say so they don't get involved that much.

And before you say it... yes I know; I want to shoot myself as well. It doesn't help that there's spaghetti code, hundreds of compiler warnings, and broken polymorphism that REALLY should be fixed, but I don't have a say in it.

Most common sort of errors I run into are null reference errors, invalid casts, and missing functions/function signature mismatches. Sometimes I am lucky and event viewer will log the class, method, and exception message. It's not the most helpful, but it's still something. The worst are the errors that have no trace, no repro steps besides a screenshot, and are generic error messages like the ones mentioned above. Sometimes it's not possible to find out why they occurred, only to pray that the environment is not properly configured, and that it will go away later.

I know this comes off as a bit of a rant, and to some extent it is. But I'm desperate for options. Are there other methods / tools I can use?

答:
Start with logging.  This will have the greatest impact.  Implement a logging framework into the code base, like Log4Net or similar. Start logging what the code does.

Debugging should be possible locally.  If not, work on getting the symbol files (PDBs) so you can debug into 3rd party dlls to gain a complete picture of the issues that are occurring.  Tools like WINDBG can point which DLLs are problematic if the system is crashing.  One can configure any server to take a memory dump when there is a crash.  It's basically a snapshot of what was going on when the problem occurred.  One can examine the dumps locally to find clues to what was happening.  If debugging is not possible, work on making it possible.  Document the steps necessary to debug.  Sometime on complex systems, there is quite a lot of setup necessary to fully debug.

Bug tracking...If you not using one, start using one.  This goes hand and hand with a proper version control system.  Basically, start tracking defects and revisions of your code.   Start to build a history of the system.

Perform Static Code Analysis.  Invest in a tool like ReSharper.  It will quickly point out all possible null reference exceptions and other bad coding practices.  It can help get the code in better shape with just a few clicks and automate tedious items like code formatting, variable naming, etc.  Measure your code, find out where the hot spots for refactoring via code metrics.

Refactor and Unit tests.  I'm going to assume that probably most of the code written is not very testable, so I wouldn't bother trying to add tests for it.  Any new code, create a tests project and start writing tests, both unit and integration.  If the unit tests fail, fail the build.  So, as you refactor, there should be tests.  One thing with tests is that one can write a test to call any method and debug into that method without loading up the entire application or code base.  This is useful to help troubleshoot issues.

Document any tribal knowledge as needed.  The code should be self documenting so comments so be sparse, but many system have "unusual" ways of doing things, point those out in a coding WIKI or other type of informal repository.  Also, consider coming up with coding standards and practices.  Enforce those standards via a toolset like Resharper.  Since most of the code is probably not following any standard and guidelines, implement the standards on new code that is written.

Since your new, I would treat this like a tour of duty.  6 months to 2 years, and then make the choice to stay or move on.  Take satisfaction from making things slightly better than the day before.


答:
For reasons of practicality, hashCode() returns a signed int. 232 possible values.  One could trivially have this be a long instead of an int and it would only slightly have me change the remainder of this answer.

The essence of all of this is the pigeonhole principle.  If you have m items and try to put them in n boxes and m > n, then you will have two items that go in the same box.

Lets look at the hashCode for Long - which can have 264 values.  Since 264 > 232 you will have two values that has to the same value.

```<code>public int hashCode() {
    return (int)(value ^ (value >>> 32));
}
</code>```

So, lets write some quick code.

```<code>System.out.println(Long.valueOf(0).hashCode());
System.out.println(Long.valueOf(-1).hashCode());
System.out.println(Long.valueOf(0).equals(Long.valueOf(-1)));
System.out.println(Long.valueOf(0).hashCode() == Long.valueOf(-1).hashCode());
</code>```

This prints out:

0
0
false
true
```

And there we have it.  Two numbers.  One of them is 0 and the other is -1 that have the same hash code, but not the same number.

The hex value of -1 is 0xffffffffffffffff which when you do an 0xffffffff ^ 0xffffffff you get 0

But wait! What if we used long for the hashCode instead!

As I said, its only a slightly different problem. I would have to start digging into another class that has the possibility of having more values than hashCodes, like BigInteger or String or Inet6Address (2128 possible values).  All it does is make the calculation of the hashCode a bit harder to do right here (those are more than a line long with two bit operations). It doesn't change that the hashCode, being a fixed size value, is subject to the pigeonhole principle.

Body:
For reasons of practicality, hashCode() returns a signed int. 232 possible values.  One could trivially have this be a long instead of an int and it would only slightly have me change the remainder of this answer.

The essence of all of this is the pigeonhole principle.  If you have m items and try to put them in n boxes and m > n, then you will have two items that go in the same box.

Lets look at the hashCode for Long - which can have 264 values.  Since 264 > 232 you will have two values that has to the same value.

public int hashCode() {
    return (int)(value ^ (value >>> 32));
}

So, lets write some quick code.

System.out.println(Long.valueOf(0).hashCode());
System.out.println(Long.valueOf(-1).hashCode());
System.out.println(Long.valueOf(0).equals(Long.valueOf(-1)));
System.out.println(Long.valueOf(0).hashCode() == Long.valueOf(-1).hashCode());

This prints out:

0
0
false
true

And there we have it.  Two numbers.  One of them is 0 and the other is -1 that have the same hash code, but not the same number.

The hex value of -1 is 0xffffffffffffffff which when you do an 0xffffffff ^ 0xffffffff you get 0

But wait! What if we used long for the hashCode instead!

As I said, its only a slightly different problem. I would have to start digging into another class that has the possibility of having more values than hashCodes, like BigInteger or String or Inet6Address (2128 possible values).  All it does is make the calculation of the hashCode a bit harder to do right here (those are more than a line long with two bit operations). It doesn't change that the hashCode, being a fixed size value, is subject to the pigeonhole principle.