问:
Learning About Different Architecture Set Ups
I'm sorry in advance if this isn't a good fit here, as it is a pretty open-ended question.

In a nutshell: where / how / when does one learn about application architecture(s)?

Normally for web applications there are several layers to the stack, generally starting with a reverse proxy or load balancer and "ending" with a DB of some sort.  Depending on requirements, there may be various caching layers involved.

There seems to be a wide variety of techniques/ tools you can set up web applications, many of the design decisions coming down to the technical requirements.  A lot of times when I read about a specific company's decisions on architecture, there isn't a lot of explanation of how they arrived at that decision, leaving that at the expense of the reader.

How do people normally go about setting up a sane architecture set-up for their specific use-case (beyond working with the technical requirements)?  Are these sort of decisions just easier to make the more experienced you are in the field?

答:
There are a variety of places one can learn about such things. There are some book-style references you can read, but a lot of information takes the form of engineers and architects speaking and writing about their systems, and bits of wisdom they have learned along the way. Some systems are very straightforward, and follow established patterns. Others solve unique problems and adopt unique architectures. It depends on what you are trying to accomplish.

Read about the libraries and platforms that architectures are built on, and read about the competitors to those subsystems (e.g. riak vs. mongo, nginx vs. apache, etc.). After a while you'll get a good picture of how things work.

I find the talks on InfoQ can be informative. 97 Things Every Software Architect Should Know has lots of small, easily digestible nuggets of information (great for reading on breaks). Lastly, High Scalability has many great articles about systems that have proven themselves to be scalable and robust.


问:
How can I track programming productivity on a daily basis?
How can I track that I'm developing software more or less productive than the previous days? </p>

答:
Both suggestions below can be roughly adopted for your need, but in both cases you need to do estimates in advance and then analyze them ad hoc (and honestly, I'm not sure if there is another effective way how to measure this, I agree with TheLQ that lines of code per time period aren't usable at all).

Agile development methodologies

Although I'm not sure how effectively it can be applied to a single developer scenario, some of the principles used in Agile can prove useful in what you aim to accomplish. Agile works in cycles in which developer(s) aim to implement stories (tasks) which are scored (in points) based on implementation complexity at the start of a development cycle, and then analyzed at the end of each cycle. This allows to determine the velocity, i.e. the number of points which a developer or a team can complete within a single development cycle.

If the way you work allows you to adopt some of the principles and organize your work in cycles, you can use the velocity per development cycle metric to track your efficiency. Note that cycles usually last 2-3 weeks, however you should be able to shorten them when using this for yourself only. It all comes down to if you can adopt such a methodology in your environment.

Evidence Based Scheduling

Even though it's primarily meant for improving estimates, you should be able to use it effectively to track decreasing trends in productivity.

ContentLicense: CC BY-SA 2.5


问:
Where to load data for things like a sidebar in MVC frameworks?
This seems extremely basic, but I've read a lot of related questions and haven't found a proper answer.

Using Ruby on Rails or other similar MVC framework, how am I supposed to load data that is displayed in sidebars on multiple pages?

Options I know of, with the problems I see:

Load it in every controller method that needs it: causes duplication of code;

Load it in a method on the ApplicationController, which is executed by all controllers: would cause data to be loaded even when not needed;

Load it directly from the model in the view: not MVC?

I've asked this elsewhere and some people told me to "look into [insert another pattern here]" or "use Ajax calls to get the data". Although those might also work, I'm looking for an answer regarding MVC frameworks on the server.

答:
Typically, I think the duplication is best handled by pushing the shared code up into a superclass (either in ApplicationController or some other superclass your controller inherits from) as a private method. I think nowadays DHH wants you to put your shared code in modules (erm, i mean concerns), and include those in your controllers, as opposed to shoving everything in ApplicationController.

Depending on your situation you'd either run the code in a before_filter on each controller action where you have your sidebar, or put the before_filter up in a superclass and call skip_before_filter when you dont want the code run before a specific controller action.

Also, its certainly possible to use ajax to accomplish this but unless you have a really good reason, I would avoid doing it that way.

There are also times when I think calling class methods on a model directly from a view is perfectly OK. Like getting simple counts, for example.

ContentLicense: CC BY-SA 3.0


问:
How to pick a platform for a new application
What should be taken into account when choosing between desktop and web as a platform for a new application?

There are definitely a lot of arguments for both platforms but what are the most important to consider?

For example:

I think that it is easier to develop a desktop application;

On the other hand it is much easier to distribute an online web application;

Cloud apps are much more popular nowadays;

If the application should interact with physical devices it would be impossible to make it as a web-app

Etc. Sorry for a possible opinion-based question

答:
@Robert Harvey gives a good rundown comparing desktop and cloud/browser based apps. To this I'd like to add a third category you might consider:

Local Web App

Run on an individual system (desktop or server)

Provide a web app interface (using HTML, JavaScript, AJAX and similar technologies)

Are generally viewed/browsed/used on the same platform they're running on (so both their server and client components are co-resident)

The web app component connects to http://locahost:SOMEPORT or its https equivalent

Gives many of the benefits of GUI apps (local residency for speed, responsiveness, and independence from network outages/performance issues; ability to interact with any local resource)

Gives many of the benefits of web apps (crafted in platform-indpendent way, not requiring platform-specific GUI libraries but instead leveraging modern web app frameworks and tools)

Having some of the drawbacks of GUI apps (esp.  if they are locally installed)

Having some of the drawbacks of web apps (e.g. somewhat lesser interactivity, dependence on browsers)

Local web apps are an interesting new application style. I've written a few, and they work nicely. They are a better bridge to cloud apps than GUIs can be, and they nicely piggyback all the year by year advances in web tooling.


问:
secure offline PC storage accessible through javascript
I'm attempting to build a browser-based HTML5 application that has the ability to store data locally on a PC (not mobile device) when offline. This data is sensitive and must be secure.

Of course the trick is trying to find a way to be able to access the secure data with Javascript.

I've ruled out browser local storage since its not secure. Could this be accomplished with a local database? If so, where could the DB credentials be stored? Javascript obviously doesn't seem like a good option to store them since its user-readable.

答:
Unfortunately, there is no way to do cryptography securely in JavaScript. To summarize key points of the link:

JavaScript has no secure random number generator and no secure key store, meaning it cannot securely generate or store encryption keys.

There is no way to guarantee the browser itself is not compromised.

Without SSL/TLS, there is no way to guarantee the data or code loaded by the browser was that provided by the server.

You can encrypt data on the server then send the data to the client over SSL/TLS. However, once the data is there, you need something to decrypt it, which requires storing the key. Unfortunately, there is no way to securely store the key. Neither local storage nor browser specific systems, e.g. chrome.storage, are encrypted.

There are other options, including ActiveX controls, Java applets, Adobe AIR or Adobe Flash apps. Unfortunately, all of these come either their own security issues or are platform specific. You could write browser plug-ins but these are also browser and sometimes platform specific.

This data is sensitive and must be secure

How secure is it? Convenience or functionality usually trumps security so you will likely have to consider the trade offs, such as:

Can you store some of the data and not the more secure parts? For example, if you are storing customer contact details, cache data that is not Personally Identifiable Information (PII, such as US social security numbers).

Can you require a certain browser or platform? For example, if most of your customers use Windows, consider an ActiveX control or a ClickOnce desktop application. If they use Windows 8+, can you create an app on the Windows app store (as much as this suggestion may make some shudder)?

Does the data change frequently, so the impact of a compromise is small?

Also remember that secure storage is not the only thing to consider. Even if you solve this issue, there is no point storing data securely if it is not delivered over SSL/TLS (so it can be viewed or tampered with in transit) or if you use weak or poor authentication (e.g. unencrypted and unhashed authentication cookie). Similarly, given physical access to the storage medium, there is no way to protect the data from a determined, patient attacker even if it is encrypted.


问:
What type of structure is this in java?
I was reading the following code

```<code>listView.setOnItemClickListener(new OnItemClickListener() {
  @Override
  public void onItemClick(AdapterView<?> parent, View view,
    int position, long id) {
    Toast.makeText(getApplicationContext(),
      "Click ListItem Number " + position, Toast.LENGTH_LONG)
      .show();
  }
});
</code>```

I am not getting what type of structure we have passed as the argument to the function setOnItemClickListener and how such type of structure works ? If it is a class then why we are using () in front of the class name and how the class functions will be called ? As i said i do not know what type of structure it is , so i do not know how to search it with proper key words on a search engine .

答:
It's a anonymous class.

setOnItemClickListener takes a instance of the type OnItemClickListener

on the first line, a instance of the anonymous class is created, the () is the constructor call

what follows inside the {} is the body of the anonymous class, which in this case, overrides the onItemClick method of OnItemClickListener

Anonymous classes are used when an implementation of a type is used only at one spot in your system.

Title: What type of structure is this in java?


答:
This is an anonymous class. One little-known feature of Java is local classes:

```<code>public void method()
{
   class MyClass{}

   MyClass cls = new myClass();
}
</code>```

This is allowed. You rarely see it, because in this form it doesn't serve any real purpose and just looks confusing. But if you want to pass a method some code to be executed, you can also use a shortcut, where you make only one instance of the class and don't give it a name (hence anonymous).

As noted in the comments, Java 8 introduces closures and lambda expressions, so that blocks of code become real language constructs that can actually be passed around. Before that, anonymous classes was the way to pass blocks of code around.

Title: What type of structure is this in java?

Body:
I was reading the following code

```<code>listView.setOnItemClickListener(new OnItemClickListener() {
  @Override
  public void onItemClick(AdapterView<?> parent, View view,
    int position, long id) {
    Toast.makeText(getApplicationContext(),
      "Click ListItem Number " + position, Toast.LENGTH_LONG)
      .show();
  }
});
</code>```

I am not getting what type of structure we have passed as the argument to the function setOnItemClickListener and how such type of structure works ? If it is a class then why we are using () in front of the class name and how the class functions will be called ? As i said i do not know what type of structure it is , so i do not know how to search it with proper key words on a search engine .

Body:
This is an anonymous class. One little-known feature of Java is local classes:

public void method()
{
   class MyClass{}

   MyClass cls = new myClass();
}

This is allowed. You rarely see it, because in this form it doesn't serve any real purpose and just looks confusing. But if you want to pass a method some code to be executed, you can also use a shortcut, where you make only one instance of the class and don't give it a name (hence anonymous).

As noted in the comments, Java 8 introduces closures and lambda expressions, so that blocks of code become real language constructs that can actually be passed around. Before that, anonymous classes was the way to pass blocks of code around.


问:
Robustness and pointer safety in C++
I'm currently building a server responsible of storing and managing few million records of fairly complex and interconnected data. For reasons beyond my control the work has to be done with C++.

I have my concerns about error handling and especially pointer safety. Dereferencing a null pointer can cause the entire server to crash whereas in some - perhaps more suitable - languages you would get a null pointer exception which you can then handle. So, basically, a bug in handling of some rare corner case of a rare operation can bring the whole service down instead of just making one particular request to fail. Bugs are impossible to avoid so how can I ensure availability and robustness in such an environment?

答:
Use C++11 (e.g. with GCC 4.9.1), not some earlier standard. Read more about memory leaks, smart pointers, buffer overflow, memory corruption, RAII.

If you follow several coding guidelines, notably the rule of three (which in C++11 is actually a rule of five), you would be able to write robust code. Use smart pointer templates like std::shared_ptr, std::unique_ptr and use standard STL containers. You  should almost never use raw pointers like SomeClass* and ::operator new (if you want to use them directly, you are probably wrong, and you should always initialize them, often to nullptr; I believe that every variable should be explicitly initialized -the compiler would optimize useless initializations-, because that makes program behavior more reproductible.).

C++11 is a difficult language. Take several weeks to read several books about it, notably by B.Stroutrup: Programming -- Principles and Practice Using C++, a Tour of C++, The C++ programming language.

Be aware that C++11 is a different language than C (and even than its predecessor C++98). You need to take time to learn it. Look also inside the source code of several recent free software projects coded in C++.

Compile with all warnings and debug info (g++ -Wall -Wextra -g). Improve your code till you got no warnings. Make an exhaustive test suite. Use valgrind (on systems where it is available). Perhaps consider using frameworks like Poco, Boost, Qt. On production machine, use a watchdog.

BTW, you might learn more about Garbage Collection (see also the GC handbook), and notice that C++ favors reference counting, which can be viewed as a simple and limited form of GC (which is unfriendly to circular references).

You could also use operating system specific things (for Linux, read Advanced Linux Programming and signal(7)...) to catch some runtime errors. (I'm not sure it is a good idea to catch SIGSEGV, you'll need some very system & processor specific code; but it is doable).

Also, you could perhaps generate some of your C++ code (see also this). My MELT system is doing so (and you might also customize GCC with MELT to add some additional coding rules checking).

You seems very concerned about null pointer dereferences. If that scares you so much (it should not, since it is actually an easy bug to find; memory leaks are harder to find), you might craft your own smart pointer (above existing facilities) which would raise an exception in that case.

BTW, robustness of your code is much more related to the code size and the care and effort put in developing and testing it, than to the amount of data or records it has to process: a well crafted 30KLOC software able to handle 1000 records would probably be able to handle many dozens of millions of records, if given enough resources (notably computer power, disk, and RAM). A few millions of records does not feel scary (consider using Sqlite or Postgresql or Mongodb, etc, etc....).

Notice that some people are able to write very robust C or C++ code; in particular, many free software database management systems (Sqlite, Postgresql, Mongodb) are coded in C or C++ and are able to run for years, managing terabytes -or perhaps petabytes- of data. If possible, make your software published as free software (and develop it in the open): you'll get useful feedback from outside.

Title: Robustness and pointer safety in C++


问:
CopyOnWriteArrayList: Why do we have to make a full copy of source data?
I have a lot of confusion about Java's CopyOnWriteArrayist. I know, when we have a modification on it like update, insert of set, it will lock the whole list, copy out a total copy of the source list, do the modification and then release the lock. So, My question is:

When doing the modification, why cannot we just lock the array, do the modification and finally release the lock? Why we have to make a new copy of the array?

The write lock is only effective for a concurrent modification, but not for a modification and read. When we are doing a read, we read from the old array and get the old data, right?

答:
1.When doing the modification, why cannot we just lock the array, do the modification and finally release the lock? Why we have to make a new copy of the array?

Because if you were to lock on write and don't create a new copy you will also have to lock on read which would be a bottleneck. You can have this functionality with Collections.synchronizedList(list). With CopyOnWriteArrayList there is no lock on read, which means is faster. Because of this CopyOnWriteArrayList is only useful when you have few updates and inserts and many concurrent reads.

2.The write lock is only effective for a concurrent modification, but not for a modification and read. When we are doing a read, we read from the old array and get the old data, right?

Yes. If you are iterating over the CopyOnWriteArrayList and while iterating a write is made concurrently, then your Iterator is not aware of the changes. You'll have to start iterating again after the update, to have the latest data.


问:
Project directory structure
Actually i'm using this project directory structure http://framework.zend.com/manual/1.12/en/project-structure.project.html

But recently i started using namespaces in all my php-classes like:

```<code>Myapp\Configuration\
Myapp\Controller\
Myapp\Persistence\Entities\
Myapp\Persistence\Dao\
Myapp\Validation
....
</code>```

and now I'm a little bit confused on where should i put all this directories in order to maintain a correct folder structure.

In the old project directory structure i also wrote directory names in lowercase, but now with namespaces i saw that most frameworks use camelcase like in the example above.

Any help would be appreciated.

答:
Most modern PHP projects are structured much like a Java package; ie a top level source folder, with subdirectories for each vendor, containing subdirectories for individual packages and so on.

However, because PHP doesn't have the same kind of loading of classes as a language like Java, it loads its classes using an autoload function. This means that files must be in a predictable location, or else be missed by the loader.

The PSR-0 and PSR-4 specifications from the PHP Framework Interop Group (PHP-FIG) are intended to standardise the stucture of packages. These specifications are adopted and implemented by pretty much all 1 modern (see: last few years) PHP libraries, packages and applications; aidedin no small part, I'm sure, by Composer.

In short, the PSR-0 specification requires that, under a given directory, files be structured and named according to the namespace of the single class they contain. For example, if the source directory is src and you have a class My\Simple\Class, then your class could be found at

```<code>~/src/My/Simple/Class.php
</code>```

1 I can't think of a single project off the top of my head, but I'm sure there are some out there.

Title: Project directory structure

Body:
Most modern PHP projects are structured much like a Java package; ie a top level source folder, with subdirectories for each vendor, containing subdirectories for individual packages and so on.

However, because PHP doesn't have the same kind of loading of classes as a language like Java, it loads its classes using an autoload function. This means that files must be in a predictable location, or else be missed by the loader.

The PSR-0 and PSR-4 specifications from the PHP Framework Interop Group (PHP-FIG) are intended to standardise the stucture of packages. These specifications are adopted and implemented by pretty much all 1 modern (see: last few years) PHP libraries, packages and applications; aidedin no small part, I'm sure, by Composer.

In short, the PSR-0 specification requires that, under a given directory, files be structured and named according to the namespace of the single class they contain. For example, if the source directory is src and you have a class My\Simple\Class, then your class could be found at

~/src/My/Simple/Class.php

1 I can't think of a single project off the top of my head, but I'm sure there are some out there.


答:
A couple of auxiliary considerations (Basile got the important stuff):

pointer errors are always the result of coding errors.

The symptom may be some unpredictable run-time crash, but the cause is contained, statically, in your code. It's entirely possible to write code that just doesn't contain these errors. The guidelines and books Basile mentions will make this easier, by teaching the right idioms and conventions.

since pointer errors are always the result of coding errors, languages which make them easy to survive just make it easy to write buggy code that doesn't actually crash. That may be fine if not crashing is the limit of your ambition, but doesn't make your code correct.

IMO a better approach for programs that need to be robust is a language that makes it hard to write incorrect code. Of course C++ isn't that either, but null pointer exceptions aren't a step in the right direction.

if you need robustness, you should build it in at all levels.

can you run multiple instances, so one failing won't take the others down?

can you split tasks into stages which can be spread across processes for the same reason?

can you journal/commit/rollback your changes cleanly? RAII-based commit/rollback works in the presence of exceptions (remember NPEs aren't the only unexpected event you should consider), and journalling can preserve changes across crashes/restarts.

if you need to handle stray cosmic rays or other bit-flipping events (as per Vatine's comment), you need ECC memory. NPEs still don't help you here: the damage is just as likely to be in your code, or the kernel code, or some filesystem metadata, and even if by some great fortune the damage occurs to a pointer, it's much more likely to become a different (and possibly valid) non-null pointer.

For reference, I don't have anything against NPEs or the languages that provide them. They're good for adding resilience to non-critical code, and for allowing a useful degree of sloppiness in applications where development time is more important than correctness.

...  in some - perhaps more suitable - languages you would get a null pointer exception which you can then handle.

This just indicates you'd like to write sloppy code and survive. While that's perfectly reasonably in some situations, that sloppy code is just as likely to silently and successfully produce garbage output as beautifully-handled NPEs.

Title: Robustness and pointer safety in C++

Body:
I'm currently building a server responsible of storing and managing few million records of fairly complex and interconnected data. For reasons beyond my control the work has to be done with C++.

I have my concerns about error handling and especially pointer safety. Dereferencing a null pointer can cause the entire server to crash whereas in some - perhaps more suitable - languages you would get a null pointer exception which you can then handle. So, basically, a bug in handling of some rare corner case of a rare operation can bring the whole service down instead of just making one particular request to fail. Bugs are impossible to avoid so how can I ensure availability and robustness in such an environment?


答:
One thing that web apps are not good at (yet, at least) is smooth interaction with a user. They don't typically use the keyboard well, and can be fiddly and frustrating with many fine mouse movements, unhelpful focus movements, network delays, UI elements jumping around with reloads, etc.

So, if your users will be using it all day and will want to use shortcuts, will want fast responses, etc....then you might be better off with a desktop application. It can even be a safety issue - you're more likely to get RSI with a mouse, especially if you use those fiddly hover-to-open menus and submenus.

Don't forget Java as a middle-ground, especially now that JavaFX 2 is around. I haven't had chance to use it yet, but it looks better than Swing from both the user and developer's point of view, and you can also embed a web browser in it (with HTML and JavaScript which can interact with the Java) which I'm hoping will be a way to make hybrid web+Java applications. And, of course, Java Webstart makes installation and updates much easier than a native application.

Title: How to pick a platform for a new application

Body:
What should be taken into account when choosing between desktop and web as a platform for a new application?

There are definitely a lot of arguments for both platforms but what are the most important to consider?

For example:

I think that it is easier to develop a desktop application;

On the other hand it is much easier to distribute an online web application;

Cloud apps are much more popular nowadays;

If the application should interact with physical devices it would be impossible to make it as a web-app

Etc. Sorry for a possible opinion-based question


问:
What to do about "stopping point syndrome"?
I have noticed a behavior in myself that I call "stopping point syndrome"... it means I cannot stop working until I reach a "stopping point" (or I become exhausted).  In other words, if I am in the zone working on getting a feature completed, I won't stop until I get it done.   If it's a bug I'm working on, this effect is even more pronounced... I won't stop until it's fixed, or at the very least, understood.  Even though I work from a laptop and have VPN and could easily go home and pick up on the work later that evening, that fact does not help me leave work... it's like I'm afraid I will die before I find the bug.

I have a very hard time explaining this to family members who can't figure out why I never come home from work at the same time and sometimes come home at 11 pm.  Even managers have been mystified by this, and I've noticed that most of my colleagues have no trouble working the same hours every day.

Does anyone else have this problem?  Should I be worried about this and/or try to change it?  If so how?

答:
It's the nature of programming to work on a longer time cycle compared to what most other people do.

A brick layer can plop another brick on in just a minute, and can stop anytime +/- one minute, and pick up where the work left off.  Simple steps, short time cycle.   In retail, it's a minute or two per customer.  A massage therapist works on a time cycle of an hour or two, and whatever was accomplished in the morning is independent of appointments for the afternoon.  A client may run over time, or show up early, but it's reasonable to expect to be done at a certain time give or take 10 or 20 minutes.

To fly a plane solo around the world to set a new world record - that's not a stop and start kind of activity.  To land every hour or two for a snack or to frolic on a  beach makes the endeavor much less impressive. It's minimum increment of time is the whole trip, whatever number of days.

It's common for a software developer to start a task, build up the mental cache of facts and details, find the bug or see how to add a new feature, and retain that cache of short-term memories until done with the coding.  You can't stop on a dime timewise.  The natural time increment that you can't break up without loss is probably anywhere from an hour to ten or so, YMMV, and there are several smaller weaker stopping points along the way where some loss of mental cache would be tolerated as well as major stopping points where the task is truly done.

Of course, our bodies tire, and one just must stop before doing more harm than good.  The old saying "death is nature's way of telling you to slow down" might apply.


问:
Refactoring web pages with user controls
Is it good design to use many user controls to help refactor a web application?

In my case, it's a VB.NET Webforms ASP.NET website. All our pages are organized into sections that, while related and belong on the same page, don't need to interact with each other. Each section contains many asp controls (text boxes, labels, grid views, buttons, etc). Most are forms the user will fill out to submit or view data. Due to the multiple sections and multiple number of controls per section, the code files can get very large.

Turning each section into a user control seems like it would be a nice way to refactor the pages. Each section now gets organized in its own file and the code gets taken out of the large page file.

We'll probably end up with 100+ user controls that aren't likely to be reused on other pages, however it seems like a better option compared to Partial Classes as it allows us to use Lazy Loading with the user controls to improve page load times.

Note: A lot of refactoring is going to happen whether we use user controls or not. If we choose to split up our page using user controls, the code that each control would contain is also going to be refactored, breaking things up into classes and functions where possible. We have the idea that user controls might be a way to help improve on our code in addition to the other refactoring as described in the question.

答:
In the real world if you are not going to re-use code/controls then do not waste your time making them re-useable. It just isn't worth the effort. However if you may re-use them in the future then make them re-usable.

You say "VB.NET Winforms ASP.NET website", however you cannot have it as Winforms and ASP.Net/Website it would have to be one or the other.


问:
Generate unique integer from two integers with algorithm
I am currently trying to come up with an algoritm which would take at least two numbers
(say user IDs) and then come up with an unique number which is generated based on these two numbers. Each integer can be as big as an INT(11) in a Database (a number with 11 digits)

Example:
User A (with ID:23, would like to interact with user B (ID: 20) and an unique number out of those
two numbers must be generated, say 4094. This number might or might not be reversible back to 23 and 20 (I don't really care)
but it's important only these two numbers to always produce this same number.
It is very important no matter of the order (23-20, or 20-23) the generated number to be still the same too).

I was thinking it would be just easier to use some hashing function
(SHA2, since MD5 allows collisions).

I also checked the suggested formula of:

π(a,b)=12(a+b)(a+b+1)+b

but it does not work when you swap A and B.

I know this can be done by a database table, which stores the
relation between the two tables but I want to avoid this by doing it
in the runtime.

Do you have any good ideas for a single math formula which would do the trick for me?

答:
It's easy to come up with a scheme, but you need a number range that is at least as big as the square of the possible number range for ids. Otherwise you cannot guarantee what you need, i.e. that the combined value is unique (google "pigeonhole principle"). This probably means that you need a bigger data type to store the combined number than for the id.

The actual solution is easy: simply format both numbers up to a fixed width and concatenate them. For instance, 35 and 534 would yield 000534000035, which equates to 534000035. It's tempting to try to build a more "efficient" scheme, but since that can't work, per the pigeonhole principle, you might as well do the most primitive thing imaginable.

Edit: I missed your requirement that the mapping must be symmetrical. For that, you must sort the two numbers consistently first.

(Note that unless you absolutely, positively have to use a numeric combined handle, it is almost certainly much easier to generate a combined string instead.)


答: If only problem is that you want the two numbers to be interchangeable, then just sort them. Make it so bigger number is always first and smaller second. Then you can use the Pairing Function you already tried.</p>

Title: Generate unique integer from two integers with algorithm

Body:
I am currently trying to come up with an algoritm which would take at least two numbers
(say user IDs) and then come up with an unique number which is generated based on these two numbers. Each integer can be as big as an INT(11) in a Database (a number with 11 digits)

Example:
User A (with ID:23, would like to interact with user B (ID: 20) and an unique number out of those
two numbers must be generated, say 4094. This number might or might not be reversible back to 23 and 20 (I don't really care)
but it's important only these two numbers to always produce this same number.
It is very important no matter of the order (23-20, or 20-23) the generated number to be still the same too).

I was thinking it would be just easier to use some hashing function
(SHA2, since MD5 allows collisions).

I also checked the suggested formula of:

π(a,b)=12(a+b)(a+b+1)+b

but it does not work when you swap A and B.

I know this can be done by a database table, which stores the
relation between the two tables but I want to avoid this by doing it
in the runtime.

Do you have any good ideas for a single math formula which would do the trick for me?


问:
Forking a repo on GitHub but allowing new issues on the fork
I have previously forked other people's repos on GitHub, and I have noticed that issues stay with the original repo, and that I can't file issues on the forked repo.

I now have the following task.  I am working for a small business where development was being done by one of the principals on his personal account.  He has amicably left the project, and we would like to migrate that project away from his personal account to a new "role" account on GitHub.

I would naturally fork the repo, in order to preserve the code history, but then I'll end up with a repo where we can't file new issues, which is quite undesirable.

How can I make a copy of this original repo into our new account, ideally still preserving code history, but be able to file new issues within this new account?

答:
This is an ancient question, and I would favor the approach that David P presents.

One more option is to remember that a local Git repository is an entire repository, complete with code history.  You could just push it as another repository on GitHub, such that GitHub would have no idea the 2 were related.  You still see your entire commit history.

This approach would cause you to loose any issue tracking history you have though.  David P's approach is superior to mine, IMO.


问:
In an Agile Environment, who is responsible for software architecture
In an agile team, who is responsible for making high-level architecture and design decisions that affect the entire system, not just the work being done in the current sprint?

Maybe product owner, scrum master, the scrum team or someone else?

答:
Architects, of course, but perhaps not the traditional kinds of architects.

I don't mean the lead-surgeon kind of architect who does all the thinking and then leaves the monkeys to do all the typing. I mean an experienced lead programmer who understands the cost/benefit tradeoffs of difficult-to-reverse decisions and who advises teams on what to do.

Effective architects are hard to find, but it can be a fantastic position, and so you probably have at least one experienced, thoughtful programmer who could do it well. Such a person needs to

Make good architecture decisions, of course, but also

Know how to give advice effectively, so that others feel comfortable taking it, and also

Slowly delegate architecture decisions to the teams

This last point trips a lot of people up. When well-meaning agilists say things like "The team owns the architecture", they have that "right", but I find the advice almost completely meaningless in its application. If you trusted teams to take responsibility for their own architecture, then you wouldn't be asking the question in the first place, now would you?! I therefore assume that you ask the question because there is some concern either that

Nobody will take responsibility and we'll have a poor architecture, or

The wrong people will take responsibility and we'll have a poor architecture, or

Whoever takes responsibility will become a scapegoat and you're hoping to avoid that

If you need someone to take responsibility, then give it to whoever wants to accept it. Seriously. That person at least cares. Give that person the resources they need to do the job and help them when they need it. It probably doesn't matter how competent the person is, because if they care, then they'll try to learn what they need to learn. Naturally, such a person needs support in the form of good books to read and a community of peers and mentors whom they can ask for help and advice.

If you worry that the wrong person is about the take responsibility, then I hope you know who "the right person" is, and will fight to install them as an architect. A book like The New Strategic Selling will help you learn the sales techniques to make that happen.

If you just need a scapegoat, then quietly nudge others to give the responsibility to whomever's career you'd most like to destroy or impair. At least be honest about it.

Getting back to an effective architect's work, they can think of their job as a management position, rather than merely a decision-making position. If they don't delegate at least the most routine decisions to the teams, then they will become a bottleneck and slow down the entire organisation. Adding more architects at the top does not make that go faster. Cultivating better architecture decisions from the ground up will. The delegation board technique will help your architect become more comfortable over time delegating more and more decisions to the teams as those teams earn trust by showing competence.

I think of a great architect as someone who helps me understand how to design my systems better, who will advise me patiently when I ask for it, and who will occasionally stop me from making a very poor decision. Such an architect acts like a leader in the truest sense of the word: someone that others voluntarily follow.

I know that was a lot.

References

Miller, The New Strategic Selling. This book includes a model for understanding why you failed to close a specific sale. I find this invaluable in understanding why coworkers won't do the obviously wonderful thing I'm suggesting.

Weinberg, Becoming a Technical Leader. This book helped me learn how to do the non-architect part of the effective architect's job.

Appelo, Delegation Boards and Delegation Poker. Don't underestimate how hard delegation is and how much we suck at it. Learn do it more effectively and more comfortably.

ContentLicense: CC BY-SA 4.0


答:
I have modified the original Cantor's formula for pairs:

pi(k1, k2) = 1/2(k1 + k2)(k1 + k2 + 1) + k2

To this custom one:

pi(k1, k2) = 1/2(k1 + k2)(k1 + k2 + 1) + (k1*k2)

So far all unique numbers, no matter f(A,B) or f(A,B). If anyone can improve the algorithm, you are welcome, I am not that good at math calculations and progression.

Here's a sample result:


问:
How to respond with error message when making AJAX calls
I have an app that wasn't written by myself. If there is an error such as an invalid ID given as a parameter, I would return something like:

(200 code)

```<code>{
    "success": 0,
    "error": "The row could not be found in the database"
}
</code>```

...but the person who wrote the application done it like so:

(404 code)

```<code>The row could not be found in the database
</code>```

I'm unsure whether their use of a 40x code is correct here. I understand 40x codes are for client errors but isn't that in the case where the file cannot be served to the user, not that the file was found and to return a error message from the script.

Anyway the 40x was causing other JavaScript to crash as it wasn't being handled correctly anyway. What is the best procedure/practise - put error messages in the response, return as 200, and handle using a success flag (e.g. "success") OR put errors in the response, return as a 40x, but handle errors from the server better in the script so not to crash other JS as what seems to be happening in Firefox for me.

Would greatly appreciate people opinions on this matter.

答:
but isn't that in the case where the file cannot be served to the user, not that the file was found and to return a error message from the script.

In principle the first part is right, but "file" is ambiguous. You're mistaken about what "file" means here, or more generally in the context of RESTful web services.

A web service exposes a resource to the user via HTTP. If the client tries to GET a resource—via AJAX or not—and the requested resource is not there, then the server should raise a 404 error. The resource is not the actual file stored on the server (e.g. ajax.php or some CGI script), but the resource the client tries to access, be it an image, or just textual content.

Wikipedia also mentions:

A 404 error should not be confused with "server not found" or similar errors, in which a connection to the destination server could not be made at all. A 404 error indicates that the requested resource may be available again in the future; however, the fact does not guarantee the same content.

Another example from A. Rodriguez, "Restful web services: The basics." IBM developerWorks (2008):

[This operation has the] effect of renaming the resource from Robert to Bob, and in doing so changes its URI to /users/Bob. In a REST Web service, subsequent requests for the resource using the old URI would generate a standard 404 Not Found error.

The AJAX submitting app should handle the case where the connection to the server times out or an internal server error (5xx) is thrown, but it should also just be able to deal with the fact that the resource it tried to fetch (e.g. a user, an image record, etc.) isn't there at this point in time.

Of course, this is all focused on RESTful approaches. You don't have to follow this, but it would make sense to leverage the semantics of HTTP status codes as much as possible, especially in the way the developer here did it—by adding specific error messages in the response body. The other approach of delivering a quasi-standard response with a success flag and a message is also fine, but it requires stronger coupling of front- and backend applications.


问:
file quantity limit in a directory on a linux file server and why?
What is a good limit to use on the quantity of files in a directory, and why?

EDIT:
Why shouldn't someone create a system that puts hundreds of thousands of files in the same directory?

Why I ask:

Someone set up a system that dumps files in a folder with a human readable date-time.

My task it to create a system that gets the files for a select time period.

Normally this wouldn't be a problem but the folder has 500,000 files in it and is growing and my system is expected to get them in real time.

Parsing 500,000 files takes too long, so I think it is the responsibility of the person who built the system that inputs the files on the FTP server to create a directory structure such as having a sub folder for each day.

答:
It depends upon the actual system (operating system, computer hardware, file system). Some [old Linux] file systems behaved badly (linear access time w.r.t. to number of directory entries). So it is generally preferable to have small sized directories of a few thousand entries at most (and this also makes the shell more happy: you might want to ls the directory). It is usually inefficient to have millions of very small files (e.g. a hundred bytes each), because often each file uses at least one file-system block.

So I would suggest having files like dir001/file001.txt .... dir123/file345.txt ....

Consider also using some other way of storing data: some indexed data file like GDBM, a Sqlite or PostGresql or MongoDb database, etc. And you might even mix the approach: use Sqlite for some meta-data about your files, and keep them in many directories. You might also have a seggregative approach: handle differently small contents and bigger ones (put the small contents in Sqlite or GDBM, and the large contents in files).


问:
Which things instantly ring alarm bells when looking at code?
I attended a software craftsmanship event a couple of weeks ago and one of the comments made was "I'm sure we all recognize bad code when we see it" and everyone nodded sagely without further discussion.

This sort of thing always worries me as there's that truism that everyone thinks they're an above average driver. Although I think I can recognize bad code I'd love to learn more about what other people consider to be code smells as it's rarely discussed in detail on people's blogs and only in a handful of books. In particular I think it'd be interesting to hear about anything that's a code smell in one language but not another.

I'll start off with an easy one:

Code in source control that has a high proportion of commented out
  code - why is it there? was it meant
  to be deleted? is it a half finished
  piece of work? maybe it shouldn't have
  been commented out and was only done
  when someone was testing something
  out? Personally I find this sort of
  thing really annoying even if it's just the odd line here and there, but when you see large blocks interspersed with the rest of the code it's totally unacceptable. It's
  also usually an indication that the rest of
  the code is likely to be of dubious
  quality as well.

答:
When there are no comments or documentation whatsoever of what the code does or is supposed to do (i.e. "the code is the documentation").

Methods or variables with a number as suffix (e.g. Login2()).


答:
I've found that there is way more repetition of code in a WebForms application than is immediately noticeable. I've been banging my head with this exact problem, and I've come up with several remedies.

DRY Up Your UserControls (And Models, Too!)

When you look at each individual page, you see so many differences that you can't possibly imagine code is being repeated. Look at the data you are saving to the database. On an application I work on, we have about 10 different forms all saving "people" records to the database with the same fields, but different fields are available in each view. At first I thought there was too much custom code, but then I realized it's all the same data in the same table. I created a generic User Control to edit "people" objects -- Populate form fields based on a domain model, and repopulate the domain model based on the form fields.

I also created a view-model that encompassed display logic, and then use that to affect the user control.

Person Domain Model

```<code>public enum PersonType
{
    Applicant = 1,
    Foo = 2
}

public class Person
{
    public int Id { get; set; }
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public string TaxId { get; set; }
    public IEnumerable<PersonType> Types { get; private set; }

    public bool IsApplicant
    {
        get { return Types.Contains(PersonType.Applicant); }
    }

    public bool IsSomethingElse
    {
        get { return Types.Contains(PersonType.Foo); }
    }

    public bool RequiresTaxId
    {
        get { return IsApplicant; }
    }
}
</code>```

Person View-Model

```<code>public class PersonViewModel
{
    public Person Person { get; private set; }

    public bool ShowTaxIdField { get { return Person.RequiresTaxId; } }

    public PersonViewModel(Person person)
    {
        Person = person;
    }
}
</code>```

(A Snippet Of) The User Control

```<code><p id="TaxIdField" runat="server">
    <asp:Label ID="TaxIdLabel" runat="server" AssociatedControlID="TaxId">Tax Id:</asp:Label>
    <asp:TextBox ID="TaxId" runat="server" />
</p>
</code>```

The C# Code-Behind

```<code>public partial class PersonUserControl : System.Web.UI.UserControl
{
    public void SetModel(PersonViewModel model)
    {
        TaxId.Text = model.Person.TaxId;
        TaxIdField.Visible = model.ShowTaxIdField;
        // ...
    }

    public PersonViewModel GetModel()
    {
        var model = new PersonViewModel(new Person()
        {
            TaxId = TaxId.Text.Trim(),
            // ...
        });

        return model;
    }
}
</code>```

Push Business Logic Into Your Domain Models

Removing duplicated code also means moving business logic into your domain models. Does this "person" require a tax Id? Well, the Person class should have a property or method called IsTaxIdRequired. Watch Crafting Wicked Domain Models for some additional ideas.

But I Use DataSets...

Stop! Immediately. You are intimately tying your C# code to the underlying database schema. Even if you have to hand write a mapping layer between DataSet objects and your domain models, you'll still be further ahead because now you can bundle behavior and business rules with your data in one neat, and tidy class. By using DataSets, your Design and Code-Behind files implement business logic! Stop this as soon as you can. Even though the WebForms framework doesn't strictly adhere to the Model-View-Controller pattern, the C# Code-Behind should be viewed as the "Controller" and the Design file should be seen as the "View". The only thing left is the "Model", which should not be a DataSet. DataSet objects give you data, but no behavior leading to repeated business logic.

Use Helper Classes For Initializing the User Interface

All to often I see this in 15 different UserControls:

```<code>public InitStateDropdownList()
{
    ddlState.DataSource = ...
    ddlState.DataBind();
    ddlState.Items.Insert(0, new ListItem("Select", string.Empty));
}
</code>```

Push this into a helper class:

```<code>public static class DropDownListHelper
{
    public static void InitStates(DrowDownList list)
    {
        list.DataSource = // get options from database or session
        list.Items.Insert(0, new ListItem("Select", string.Empty));
        list.DataBind();
    }
}
</code>```

And use it:

```<code>DropDownListHelper.InitStates(ddlState);
</code>```

Some people rile against "helper" classes, but it's surely better than writing what is essentially the same 3 lines of code in multiple places. Even if you can't create super generic User Controls, at least weed out the repetitive code that initializes components upon page load.

When is the BIG Rewrite the Answer?

Almost never. For all my complaints about the WebForms framework, it at least compartmentalizes pages and forms, which is a great setup for a long term refactoring job. The key is adding test coverage. I've had a lot of success using CodedUI Tests and SpecFlow for testing WebForms applications, since most if not all business logic is scattered between C# and ASP in 19 different files. At least write some tests to validate existing business rules don't get broken during the many refactoring sessions you have in your future.

Title: Refactoring web pages with user controls

Body:
Is it good design to use many user controls to help refactor a web application?

In my case, it's a VB.NET Webforms ASP.NET website. All our pages are organized into sections that, while related and belong on the same page, don't need to interact with each other. Each section contains many asp controls (text boxes, labels, grid views, buttons, etc). Most are forms the user will fill out to submit or view data. Due to the multiple sections and multiple number of controls per section, the code files can get very large.

Turning each section into a user control seems like it would be a nice way to refactor the pages. Each section now gets organized in its own file and the code gets taken out of the large page file.

We'll probably end up with 100+ user controls that aren't likely to be reused on other pages, however it seems like a better option compared to Partial Classes as it allows us to use Lazy Loading with the user controls to improve page load times.

Note: A lot of refactoring is going to happen whether we use user controls or not. If we choose to split up our page using user controls, the code that each control would contain is also going to be refactored, breaking things up into classes and functions where possible. We have the idea that user controls might be a way to help improve on our code in addition to the other refactoring as described in the question.

Body:
I've found that there is way more repetition of code in a WebForms application than is immediately noticeable. I've been banging my head with this exact problem, and I've come up with several remedies.

DRY Up Your UserControls (And Models, Too!)

When you look at each individual page, you see so many differences that you can't possibly imagine code is being repeated. Look at the data you are saving to the database. On an application I work on, we have about 10 different forms all saving "people" records to the database with the same fields, but different fields are available in each view. At first I thought there was too much custom code, but then I realized it's all the same data in the same table. I created a generic User Control to edit "people" objects -- Populate form fields based on a domain model, and repopulate the domain model based on the form fields.

I also created a view-model that encompassed display logic, and then use that to affect the user control.

Person Domain Model

public enum PersonType
{
    Applicant = 1,
    Foo = 2
}

public class Person
{
    public int Id { get; set; }
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public string TaxId { get; set; }
    public IEnumerable<PersonType> Types { get; private set; }

    public bool IsApplicant
    {
        get { return Types.Contains(PersonType.Applicant); }
    }

    public bool IsSomethingElse
    {
        get { return Types.Contains(PersonType.Foo); }
    }

    public bool RequiresTaxId
    {
        get { return IsApplicant; }
    }
}

Person View-Model

public class PersonViewModel
{
    public Person Person { get; private set; }

    public bool ShowTaxIdField { get { return Person.RequiresTaxId; } }

    public PersonViewModel(Person person)
    {
        Person = person;
    }
}

(A Snippet Of) The User Control

<p id="TaxIdField" runat="server">
    <asp:Label ID="TaxIdLabel" runat="server" AssociatedControlID="TaxId">Tax Id:</asp:Label>
    <asp:TextBox ID="TaxId" runat="server" />
</p>

The C# Code-Behind

public partial class PersonUserControl : System.Web.UI.UserControl
{
    public void SetModel(PersonViewModel model)
    {
        TaxId.Text = model.Person.TaxId;
        TaxIdField.Visible = model.ShowTaxIdField;
        // ...
    }

    public PersonViewModel GetModel()
    {
        var model = new PersonViewModel(new Person()
        {
            TaxId = TaxId.Text.Trim(),
            // ...
        });

        return model;
    }
}

Push Business Logic Into Your Domain Models

Removing duplicated code also means moving business logic into your domain models. Does this "person" require a tax Id? Well, the Person class should have a property or method called IsTaxIdRequired. Watch Crafting Wicked Domain Models for some additional ideas.

But I Use DataSets...

Stop! Immediately. You are intimately tying your C# code to the underlying database schema. Even if you have to hand write a mapping layer between DataSet objects and your domain models, you'll still be further ahead because now you can bundle behavior and business rules with your data in one neat, and tidy class. By using DataSets, your Design and Code-Behind files implement business logic! Stop this as soon as you can. Even though the WebForms framework doesn't strictly adhere to the Model-View-Controller pattern, the C# Code-Behind should be viewed as the "Controller" and the Design file should be seen as the "View". The only thing left is the "Model", which should not be a DataSet. DataSet objects give you data, but no behavior leading to repeated business logic.

Use Helper Classes For Initializing the User Interface

All to often I see this in 15 different UserControls:

public InitStateDropdownList()
{
    ddlState.DataSource = ...
    ddlState.DataBind();
    ddlState.Items.Insert(0, new ListItem("Select", string.Empty));
}

Push this into a helper class:

public static class DropDownListHelper
{
    public static void InitStates(DrowDownList list)
    {
        list.DataSource = // get options from database or session
        list.Items.Insert(0, new ListItem("Select", string.Empty));
        list.DataBind();
    }
}

And use it:

DropDownListHelper.InitStates(ddlState);

Some people rile against "helper" classes, but it's surely better than writing what is essentially the same 3 lines of code in multiple places. Even if you can't create super generic User Controls, at least weed out the repetitive code that initializes components upon page load.

When is the BIG Rewrite the Answer?

Almost never. For all my complaints about the WebForms framework, it at least compartmentalizes pages and forms, which is a great setup for a long term refactoring job. The key is adding test coverage. I've had a lot of success using CodedUI Tests and SpecFlow for testing WebForms applications, since most if not all business logic is scattered between C# and ASP in 19 different files. At least write some tests to validate existing business rules don't get broken during the many refactoring sessions you have in your future.


答: Don't ever make a user control that will only get used once, all it will do is hide the real problems and make maintaining things even more difficult. If you have huge code files from the sheer number of controls on a page, you need to make more separate pages not user controls. It already sounds like you have multiple sections on a page that could relatively easily be totally separate pages.</p>


问:
Why are floating point numbers used often in Science/Engineering?
While investigating the accuracy of floating point numbers, I've seen in a few places a statement similar to

"float and double are (designed for/used often in) engineering and scientific calculation"

From my understanding, the strength of floats and doubles are the amount of memory they use for their (good, but not perfect) precision.

I feel like I'm almost getting an understanding from this answer

"floating point numbers let you model continuous quantities"

I still am not convinced I understand. Engineering and Science both sound like fields where you would want precise results from your calculations, which, from my understanding, floating points do not give. I'm also not sure I follow what a "continuous quantity" is, exactly.

Can somebody expand on this explanation and perhaps give an example?

答:
Computation in science and engineering requires tradeoffs in precision, range, and speed. Fixed point arithmetic provides precision, and decent speed, but it sacrifices range. BigNum, arbitrary precision libraries, win on range and precision, but lose on speed.

The crux of the matter is that most scientific and engineering calculations need high speed, and huge range, but have relatively modest needs for precision. The most well determined physical constant is only known to about 13 digits, and many values are known with far less certainty. Having more than 13 digits of precision on the computer isn't going to help that. The fly in the ointment is that sequences of floating point operations can gradually lose precision. The bread and butter of numerical analysis is figuring out which problems are particularly susceptible to this, and figuring out clever ways of rearranging the sequence of operations to reduce the problem.

An exception to this is number theory in mathematics which needs to perform arithmetic operations on numbers with millions of digits but with absolute precision. Numerical number theorists often use BigNum libraries, and they put up with their calculations taking a long time.


答:
Note that floating point numbers are basically the same as scientific and engineering notation, the standard way for humans to write numbers in math and science. In these fields, there's not a great need for extreme precision, but there's often a huge range.

To pick a random example from my physics homework, I recently had to work with the mass of an electron, which is roughly 9.11 * 10^-31 kg. I don't care a lot about the precision; it could easily be 9.12 for all I care. But I care about the exponent and don't want to have to write out 0.0000...911 kg, so I use scientific notation.

Similar reasoning applies in scientific and engineering computing: there's a huge range, but we don't want to have to store and work with very large numbers, so we store a normalized value and an exponent, which is smaller and faster to work with.

Title: Why are floating point numbers used often in Science/Engineering?

Body:
While investigating the accuracy of floating point numbers, I've seen in a few places a statement similar to

"float and double are (designed for/used often in) engineering and scientific calculation"

From my understanding, the strength of floats and doubles are the amount of memory they use for their (good, but not perfect) precision.

I feel like I'm almost getting an understanding from this answer

"floating point numbers let you model continuous quantities"

I still am not convinced I understand. Engineering and Science both sound like fields where you would want precise results from your calculations, which, from my understanding, floating points do not give. I'm also not sure I follow what a "continuous quantity" is, exactly.

Can somebody expand on this explanation and perhaps give an example?


问:
Model relationships with DDD (or with sense)?
Here is a simplified requirement:

User creates a Question with multiple Answers. Question must have at least one Answer.

Clarification: think Question and Answer as in a test: there is one question, but several answers, where few may be correct. User is the actor who is preparing this test, hence he creates question and answers.

I am trying to model this simple example so to 1) match the real life model 2) to be expressive with the code, so to minimize potential misuse and errors, and to give hints to developers how to use the model.

Question is an entity, while answer is value object. Question holds answers. So far, I have these possible solutions.

[A] Factory inside Question

Instead of creating Answer manually, we can call:

```<code>Answer answer = question.createAnswer()
answer.setText("");
...
</code>```

That will create an answer and add it to the question. Then we can manipulate answer by setting it's properties. This way, only questions can crete an answer. Also, we prevent to have an answer without a question. Yet, we do not have control over creating answers, as that is hardcoded in the Question.

There is also one problem with the 'language' of above code. User is the one who creates answers, not the question. Personally, I don't like we create value object and depending on developer to fill it with values - how he can be sure what is required to add?

[B] Factory inside Question, take #2

Some says we should have this kind of method in Question:

```<code>question.addAnswer(String answer, boolean correct, int level....);
</code>```

Similar to above solution, this method takes mandatory data for the answer and creates one that will be also added to the question.

Problem here is that we duplicate the constructor of the Answer for no good reason. Also, does question really creates an answer?

[C] Constructor dependencies

Let's be free to create both objects by our selves. Let's also express the dependency right in constructor:

```<code>Question q = new Question(...);
Answer a = new Answer(q, ...);   // answer can't exist without a question
</code>```

This gives hints to developer, as answer can't be created without a question. However, we do not see the 'language' that says that answer is 'added' to the question. On the other hand, do we really need to see it?

[D] Constructor dependency, take #2

We can do the opposite:

```<code>Answer a1 = new Answer("",...);
Answer a2 = new Answer("",...);
Question q = new Question("", a1, a2);
</code>```

This is opposite situation of above. Here answers can exist without a question (which does not make sense), but question can not exist without answer (which make sense). Also, the 'language' here is more clear on that question will have the answers.

[E] Common way

This is what I call the common way, the first thing that ppl usually do:

```<code>Question q = new Question("",...);
Answer a = new Answer("",...);
q.addAnswer(a);
</code>```

which is 'loose' version of above two answers, since both answer and question may exist without each other. There is no special hint that you have to bind them together.

[F] Combined

Or should I combine C, D, E - to cover all the ways how relationship can be made, so to help developers to use whatever is best for them.

Question

I know people may choose one of the answers above based on the 'hunch'. But I wonder if any of above variant is better then the other with a good reason for that. Also, please don't think inside the above question, I would like to squeeze here some best practices that could be applied on most cases - and if you agree, most use cases of creation some entities are similar. Also, lets be technology agnostic here, eg. I do not want to think if ORM is going to be used or not. Just want good, expressive mode.

Any wisdom on this?

EDIT

Please ignore other properties of Question and Answer, they are not relevant for the question. I edited above text and changed the most of the constructors (where needed): now they accept any of necessary property values needed. That may be just a question string, or map of strings in different languages, statuses etc. - whatever properties are passed, they are not a focus for this ;) So just assume we are above passing necessary parameters, unless said different. Thanx!

答:
In case where requirements are so simple, that multiple possible solutions exist, then KISS principle should be followed. In your case, that would be option E.

There is also case of creating code that expresses something, that it should not. For example tying creation of Answers to Question (A and B) or giving Answer reference to Question (C and D) add some behavior that is not necessary to the domain and might be confusing. Also, in your case, Question would most probably be aggregate with Answer and Answer would be a value type.


答:
Floating-point numbers also have several properties that lend themselves well to computing certain types of scientific results. Most notably, precision is inversely proportional to magnitude, just like in scientific notation, so you can represent both small differences close to zero and larger differences much farther away.

Goldberg's paper is probably the most famous analysis of the properties of floating-point numbers (and should be required reading if you care about this sort of thing), but Kahan's papers I think do a better job of explaining the rationale behind many of the subtle design issues.

In particular, Kahan's diatribe about Java's implementation of floating point, while quite inflammatory, makes several good points about why IEEE-754 semantics are useful, and Much Ado About Nothing's Sign Bit explores the rationale for signed zero in considerable depth.


答:
Your proposition about science is wrong, Engineering and Science other then Math don't work with exact precise results. They work with a precision factor which is built into how many digits you show.

The key term you need to understand here is:  significant figures.
The significant figures of a number are those digits that carry meaning contributing to its precision.

Which basically means if I state that something is 12 centimeters longs, it can actually be somewhere between 11,5 and  12,5 centimeters long.
If however I state that something is 12,00 centimeters  long it can be somewhere between 11,995 and 12,005 centimeters long.

Just as an illustration, if you take a measurement tape and measure your living room. Even though you may find that it 6 meters 25 centimeters wide, you know that your tape measurement wasn't accurate enough to tell anything about the millimeter-accuracy or nano-meter-accuracy.


问:
Application qos involving priority and bandwidth
Our manager wants us to do applicaiton qos which is quite different from the well-known system qos. We have many services of three types, they have priorites, the manager wants to suspend low priority services requests when there are not enough bandwidth for high priority services. But if the high priority services requests decrease, the bandwidth for low priority services should increase and low priority service requests are allowed again.

There should be an algorithm involving priority and bandwidth. I don't know how to design the algorithm, can anyone assist in getting this right?

All these services are within a same process. We are setting the maximum bandwidth for the three types of services via ports of services via TC (TC is the linux qos tool whose name means traffic control).

答:
The first thing to understand is that you're basically needing to build a subset of all the parts that go into an operating system scheduler, including:

instrumentation and monitoring (What resources are currently available?)

Decision making / policy (How should resources be allocated? How will the users or administrators communicate their policy / priority desires?)

Enforcement (How should processes be restricted from using more than their allocated amount?)

The good news is that a lot of the components, especially in the monitoring and enforcement realms, will be very thin layers on existing mechanisms such as as Linux cgroups, netstat, and tc.

There is even a chance that the policy mechanism of cgroups will be sufficient to your need. It's a lot more sophisticated than the "priorities" of historical Unix, and many commercial shops have relied on similar (albeit proprietary / single vendor) facilities such as HP PRM and AIX WPARs for application QoS management.

These typically work on some variant of fair share scheduling, which allows one to define relative priorities for different classes or groups of jobs/processes, and arrange the classes/groups with respect to one another, so that their relative importance and resource allocations are as the policy or "as the business" intended, yet without completely starving the lower-priority jobs. (If you're willing to starve lower-priority jobs completely, simpler mechanisms can be used.)

Simple forms of FCS use single-level classes, with each member in a class being equal to the others. More refined forms use hierarchical groupings, so that the "minimum 50% resources devoted to Platinum" users can be further subdivided with specific allocations or minimums for subgroups or even individual users. (Having a single app/job be the only member of a group is an easy way to give it the group's entire allocation.) This page goes into some detail about how one of these "workload managers" (Adaptive Computing's Moab) operates.

There are a variety of ways the policies can be defined. "Shares" is a relatively sophisticated way, but one that novice users often find difficult. "Percentages" or "priorities" are somewhat less precise and more perturbable, but can be more popular with less mathematical policy-setters.

And there are a lot of nuances possible. The LPAR allocations of AIX and IBM Power machines, for example, have a number of subtle options for temporary priority inversions, "stealing" cycles that would otherwise go unused, and otherwise tweaking the policies. If you want something very sophisticated, proprietary Unix systems have a number of good study points.


问:
Moving old desktop application to a robust platform
Several years ago I wrote a desktop application, a small accounting system, in Delphi 7, that is in user in a medium-sized company.
The code belongs to me. I ported another Accounting system that I had built years before (Delphi/IB), to use SQL Server and added new business rules, in my free time, then I gave the application to the company. (No contracts, no license, simply the code is mine,  no one is arguing that).

It currently works by connecting directly to the database. Each user in the application is a user in the database. I have roles defined for each form the user is able to see, and also they matches DB roles in SQL Server, so, not even by mistake you would be able to see data you shouldn't be.
I also managed to mangle the passwords so that the user (a normal user, not a hacker) won't ever be able to connect directly to the DB even if it knows the ip/port of the server.
It has worked in this way for years.

A new IT management, lots of new people, are frown upon my application.
Take note that this company also use SAP for many of the branches and my accounting system servers the purpose of a sort of a consolidation system.
They don't like the Client/Server architecture and are pointing fingers to security risk and what not. I agree my C/S architecture and the way I connect to the database doesn't feel robust.

I suppose I need "something" in the middle. But researching for 3-tier application I'm drowned in a sea of articles about 3-tier in the same application, I mean, just a separation of data within the same application, while I think what I need is a 2nd layer of software between my desktop app and the database. I guess I would need an application server too, may be?

I'm also fluent in C# and lately I have written more desktop apps in C# then in Delphi, so I could migrate my Delphi code to C# if needed. Since the code is mine, this is sort of an investment, I might capitalize in the future. That's why, besides the request from IT, I'm also interested in making this more robust.

I'm looking for an article to get me started in order to write such piece of software.
WCF, Midas, ORM?

Without wanting to look pretentious, I'm looking forward to create a robust platform, like that used in SAP-ERP. So I could grow the application adding new modules, like controlling, payroll, billing, etc and turn it into a some sort of small ERP.

答:
Not clear whether you are saying your CIO's issue with your design is the persistent DB connection, or the fact that you use application users mapped to DB users. Neither is necessarily BAD architecture, depending on the requirements. There are many applications that do the same thing, even high end stuff that runs on Oracle Enterprise.

And, even if you eliminate the persistent, direct connection, a competent admin (much less hacker) running your app locally, with or without access to the DB server will be able to connect directly to the database if they work hard enough. Why is that a problem? Sure you don't want it to be public facing, but internal apps can be managed with firewall rules, or co-resident app + db on the same server. Traditionally, if we embed a database engine in an app we don't change the port. The security can't be based on obscurity. Either make sure the app is co-resident so there is no DB network connection except localhost, or ensure it is using an encrypted session, that removes the issue of network snooping, but leaves local hacking possibilities. Outside of that, what is the problem?

Are you not able to dedicate a database instance to the app? Perhaps he wants your app to co-exists on a SQL Server instance with other apps. If so, I can understand the requirement, and it is a good thing. Your choice to map users to real DB users was not a great choice, but it isn't end of the world, that is more of an old school design that works, but limits your options (requires dedicated database). Now that it works, you have to weigh the costs. Maybe your CIO already has.

Moving on, a local server (local LAN) for locally installed desktop application is the most common model. Otherwise, you either move to a full SaaS web model, or use disconnected synchronization. It isn't an OLTP web server so I don't understand the issue with persistent sessions. If he is stuck on this point, there is still some good in rearchitecting it...here are some options, with benefits.

You can port a connected SQL design to a disconnected SOA design by creating a service "agent" (WCF services serving up data calls). You need to map all of your queries over to service calls, implement them in a service layer, then deploy that service layer as the front-end to your database. You can still use fat client data grids all you like, but the data flows via WCF or Data Services from app to broker, broker to DB, and if you need to get fancy, you can implement IQueryable and your own custom data source to allow arbitrary query criteria. I'm not saying it is painless and cheap to transition, but it works. The advantage is you can move your data server anywhere on the Internet (cloud) and your fat app can still work over HTTP/HTTPS. C# / .NET is perfect for this. It's service interoperability is very rich.

If your DB is not local on a LAN/WAN, and performance/latency is an issue, there are other options that I've also had success with, such as implementing a local SQLAnywhere database, replicate the schema of the master DB, and use Mobilink to synchronize against the consolidated database. There are a lot of commercial products with embedded SQLAnywhere or Ultalite DBs, or SQLLite DBs that use this architecture. (Hate to give away tricks of my trade but...)


答:
Sorry, but this is madness. Rewriting a working application in a different language because someone doesn't like the style of data access plumbing?! It's needless expense, effort, disruption, and risk. Where's the ROI? What is the time-to-value? If the question describes the full reasoning, standard go/no-go business metrics cannot possibly support rip-and-replace.

This isn't because I disagree. Direct-to-the-database connections have some real downsides, such as limited scalability, and likely violation of the least-privilege security approach. They lock data interpretation into client apps, making multi-platform apps, long version tails, and combination tasks such as data analytics more challenging. And Delphi 7 is getting a little long in the tooth. Etc. etc.

If you were a commercial software company, needed to greatly increase the number of concurrent users, needed to operate with less trusty users or on less secure networks, or had a specific feature need that direct-to-database connections impeded or precluded, I'd be the first to concur with "update your architecture" or "modernize your tooling." But from your question at least, those don't seem to be the case. You are a medium-sized company; the software appears to be doing its job; and there is no clear motivation for a clean slate rebuild or major roto-tilling. That makes this a major update to conform to someone's architectural preferences. That violates primum non nocere ("first, do no harm") for questionable benefit.

This is exacerbated by the likelihood that you don't have good test suites to confirm that a new clean-piece-of-paper build works correctly. (Don't feel too bad about that--most shops don't for their internal apps.) And if it's accounting software, getting data wrong can have extremely negative business consequences. All of this shouts "this will end badly!"--or at least, "you will eventually be unhappy that you decided to rebuild something that already worked for shaky reasons, rather than work on something that could genuinely help the company." I've seen too many clean-slate projects end that way.

Major roto-tilling or start-fresh can work--but you have to have substantial backing, management and users that accept it will take a while and introduce disruption along the way, and most important, a solid reason for going that way.

Instead, I'd suggest you examine what incremental update paths you might have from Delphi 7, and develop a more incremental plan and a more benefits-based motivation for making substantial changes.

Title: Moving old desktop application to a robust platform

Body:
Several years ago I wrote a desktop application, a small accounting system, in Delphi 7, that is in user in a medium-sized company.
The code belongs to me. I ported another Accounting system that I had built years before (Delphi/IB), to use SQL Server and added new business rules, in my free time, then I gave the application to the company. (No contracts, no license, simply the code is mine,  no one is arguing that).

It currently works by connecting directly to the database. Each user in the application is a user in the database. I have roles defined for each form the user is able to see, and also they matches DB roles in SQL Server, so, not even by mistake you would be able to see data you shouldn't be.
I also managed to mangle the passwords so that the user (a normal user, not a hacker) won't ever be able to connect directly to the DB even if it knows the ip/port of the server.
It has worked in this way for years.

A new IT management, lots of new people, are frown upon my application.
Take note that this company also use SAP for many of the branches and my accounting system servers the purpose of a sort of a consolidation system.
They don't like the Client/Server architecture and are pointing fingers to security risk and what not. I agree my C/S architecture and the way I connect to the database doesn't feel robust.

I suppose I need "something" in the middle. But researching for 3-tier application I'm drowned in a sea of articles about 3-tier in the same application, I mean, just a separation of data within the same application, while I think what I need is a 2nd layer of software between my desktop app and the database. I guess I would need an application server too, may be?

I'm also fluent in C# and lately I have written more desktop apps in C# then in Delphi, so I could migrate my Delphi code to C# if needed. Since the code is mine, this is sort of an investment, I might capitalize in the future. That's why, besides the request from IT, I'm also interested in making this more robust.

I'm looking for an article to get me started in order to write such piece of software.
WCF, Midas, ORM?

Without wanting to look pretentious, I'm looking forward to create a robust platform, like that used in SAP-ERP. So I could grow the application adding new modules, like controlling, payroll, billing, etc and turn it into a some sort of small ERP.


问:
How long are scrum meetings supposed to last?
I read that Scrum meetings should be less than 15 minutes long. Is this how long they usually are, or do they depend on the size of the development team? Must every team member speak in these?</p>

答:
Your team is probably too big. Various sources, Mountain Goat Software among them, recommend doing team(s) of teams (Scrum of Scrums). This is analogous to the pizza rule that Amazon uses. The core principle here is to reduce the number of connections between people so that communication is less complex.

Fixing this would effectively require that you figure out how to break up the big team into smaller teams and have the authority or ability to persuade the authority to adopt this method.


问:
Best approach to a defined datastructure
I have a plain text file with some data that I can't change, so I have to create a datastructure to use in reading the file and with that datastructure do some thousands of interactions.

The file follows a defined pattern like this:

```<code>AAAAAA:zz_something sometxt1 sometxt2
AAAAAA:zz_something sometxt1 sometxt2
AAAAAA:zz_someotherthing sometxt1 sometxt2
BBBBBB:zz_something sometxt1 sometxt2
BBBBBB:zz_someotherthing sometxt1 sometxt2
BBBBBB:zz_something sometxt1 sometxt2
CCCCC:zz_others blahblah blehbleh
DDDDD:blihblih blohbloh
EEEEE:bluhbluh
</code>```

This text is divided basically in three parts (most of it)

first, before the :, would be my object1

second, after the : AND IF it starts with zz_, would be my object2

third, everything else from that point on would be a List of strings

for the part after : that does not start with zz_ I will use a default string to identify it.

At first I thought of creating three objects with this structure:

```<code> obj1                       obj2                 obj3
    String id                 String id            String str1
    Set<String> obj2          List<obj3> list      String str2
</code>```

Plus a main object that would hold a Set<Obj1> main. But when I start to implement this structure I realize that it would be very difficult to read this in a loop to get any element inside it, like main.get('obj1').get('obj2').list, because it is a Set and I cannot do that.

So I ended up using a map as Map<String, Map<String, List<String>>> main but I feel like there must be a better way to do this.

What would be your recomendations?

答:
I come from a perl background.  There, the combination of having autovivification and easy nested hash/list structures... the nested maps don't look too bad if you can keep your head around them.  But on the other hand, they aren't that great either and it's really easy for a null pointer to creep in with other languages.  In more formal languages, that you display your implementation makes refactoring a nightmare.  It's certainly less than optimal or idiomatic Java code.

So, we've got three objects.  They're nested.  And that, while it can make it a bit awkward, you might want to look at actually making them nested classes.

```
One    Two          Three
AAAAAA:zz_something sometxt1 sometxt2
AAAAAA:zz_something sometxt1 sometxt2
AAAAAA:zz_someotherthing sometxt1 sometxt2
```

In this example, you would have a class One, Two, and Three.  Class One would have an associated id of 'AAAAAA' and then other stuff.  So you could have a definition akin to:

```<code>class One {
    String id;
    Map<String, Two> data;

    public void One() { id = null; data = new HashMap(); }

    public void build(String line) throws IllegalArgumentException {
        parse line;
        if (id == null) { id = parsedId; }
        else { throw new IllegalArgumentException("attempt to reset id"); }
        if data.contains(twoId) {
            two = data.get(twoId);
        } else {
            two = new One.Two();
            data.put(twoId, two);
        }
        two.build(line);
    }

    public static class Two {
        // ....
        private void build(String line) {
            // ....
            three = new One.Two.Three();
            three.build(line);
        }
    }
}
</code>```

Note that the Two.build method is private.  Because it's a nested class, the outer class can access the inner class private members and methods.  This allows you to restrict the building of Two (and Three) to the public interface that One provides.  This prevents other code from tinkering with it and messing it up later.

Furthermore, while this doesn't have all the access methods, they would be provided via the public interface of One in accordance with the Law of Demeter.

Yes, underneath this, this is a Map<String, Map<String, List<String>>> - because that is what the data is.  However, its one that doesn't disclose its implementation, or make a mess.  Its still up to you to write methods of one.find("sometxt2") which returns zz_something if you want it, but it makes it much easier for the person using the object to do so. This in turn makes them less likely to make errors with the code (even if that person is you) and reduces the cognitive load allowing faster development and less worry about what is going on.

This also encourages dry code in that you won't be writing the code for find(String arg) again and again and again each time you need it... or writing a 'helper' class with static methods that work on the data.  When you find yourself writing SomeDataHelper classes or SomeDataUtil classes, you've likely got a class in there that wants to escape into a form akin to the one described above.

As an alternative to the nested class, one could use a package to hold the three classes and make the methods package protected (default protection - not public, or protected, or private) instead.  This could make serialization easier and avoid what could very well be a java file that is three times longer (its three classes).

Body:
I come from a perl background.  There, the combination of having autovivification and easy nested hash/list structures... the nested maps don't look too bad if you can keep your head around them.  But on the other hand, they aren't that great either and it's really easy for a null pointer to creep in with other languages.  In more formal languages, that you display your implementation makes refactoring a nightmare.  It's certainly less than optimal or idiomatic Java code.

So, we've got three objects.  They're nested.  And that, while it can make it a bit awkward, you might want to look at actually making them nested classes.

One    Two          Three
AAAAAA:zz_something sometxt1 sometxt2
AAAAAA:zz_something sometxt1 sometxt2
AAAAAA:zz_someotherthing sometxt1 sometxt2

In this example, you would have a class One, Two, and Three.  Class One would have an associated id of 'AAAAAA' and then other stuff.  So you could have a definition akin to:

class One {
    String id;
    Map<String, Two> data;

    public void One() { id = null; data = new HashMap(); }

    public void build(String line) throws IllegalArgumentException {
        parse line;
        if (id == null) { id = parsedId; }
        else { throw new IllegalArgumentException("attempt to reset id"); }
        if data.contains(twoId) {
            two = data.get(twoId);
        } else {
            two = new One.Two();
            data.put(twoId, two);
        }
        two.build(line);
    }

    public static class Two {
        // ....
        private void build(String line) {
            // ....
            three = new One.Two.Three();
            three.build(line);
        }
    }
}

Note that the Two.build method is private.  Because it's a nested class, the outer class can access the inner class private members and methods.  This allows you to restrict the building of Two (and Three) to the public interface that One provides.  This prevents other code from tinkering with it and messing it up later.

Furthermore, while this doesn't have all the access methods, they would be provided via the public interface of One in accordance with the Law of Demeter.

Yes, underneath this, this is a Map<String, Map<String, List<String>>> - because that is what the data is.  However, its one that doesn't disclose its implementation, or make a mess.  Its still up to you to write methods of one.find("sometxt2") which returns zz_something if you want it, but it makes it much easier for the person using the object to do so. This in turn makes them less likely to make errors with the code (even if that person is you) and reduces the cognitive load allowing faster development and less worry about what is going on.

This also encourages dry code in that you won't be writing the code for find(String arg) again and again and again each time you need it... or writing a 'helper' class with static methods that work on the data.  When you find yourself writing SomeDataHelper classes or SomeDataUtil classes, you've likely got a class in there that wants to escape into a form akin to the one described above.

As an alternative to the nested class, one could use a package to hold the three classes and make the methods package protected (default protection - not public, or protected, or private) instead.  This could make serialization easier and avoid what could very well be a java file that is three times longer (its three classes).


答:
The Scrum meetings should mainly involve

What is the progress from the last meeting?

What do you plan to do next?

Do you see any blockers?

The actual time depends on the team size and should be short and quick.
This is mainly a technique in agile world where you want to dynamically assess the situation and take decisions and also increase the transparency and better communication.

Title: How long are scrum meetings supposed to last?

Body: I read that Scrum meetings should be less than 15 minutes long. Is this how long they usually are, or do they depend on the size of the development team? Must every team member speak in these?</p>


问:
Efficient database access for related data
As an example for this question, consider a content management system that has a content database table which contains things like the page title and location as well as the content itself which could be quite a large amount of text. There is a second database table with details of resources (such as images and files) that are associated with a content item. Each page in the content management system may have one or many content items, each with it's own resources.

At present I am retrieving a list of content items and then for each content item doing a second query to retrieve the list of related resources. On a page with 10 content items, this means 11 queries.

I could do a join and retrieve all content and related resources in one query, but it concerns me that where there are multiple resources for a content item, the content item will be duplicated and, because the content item might be large, this might be very inefficient with so much duplicate data.

Another option is to have a stored procedure that returns two recordsets. The first recordset contains the content, and the second recordset contains the related resources for all of the content items in the first recordset, which is separated out by the receiving server.

So the question is what is generally the most efficient way to retrieve data in this situation?

I know that performance testing is often the answer but for this question I was looking for best practise type advice or alternatives that I may not have thought of.

答:
"Don't worry. Be happy." Pretty much any way you approach this problem is going to have adequate performance for small through medium sized CMSs.

Diving In

Do you really need to query for each content item? CMSs like WordPress often store their pages in HTML format. Pages may reference images, audio, video, and other multimedia, but they usually do so from the HTML, which means that when displaying the page, one never really needs to query for the associated references.

In the naïve case, content items are simply loaded, based on independent HTTP requests, from the local file system. In more sophisticated setups, multimedia is loaded from a separate "static" content server or (better yet), a content delivery network.

Probably every web server on the planet has features or add-ons to support this kind of "split serving," and more performance-oriented servers like nginx eagerly support this model (along with extensive in-the-field experience being deployed alongside other accelerators such as caching servers like Squid and Varnish.

When editing a page, there is a greater likelihood one would want or need to query all associated content. That is the one case where querying multiple times might make more sense. But, this is much more likely to result in 2 queries than 11--one for the page, and then one omnibus query that returns 10 results to a query for all associated content.

There are a number of schemes for accomplishing this in SQL, including storing a list of content ids in each page, storing the "associated page item" in each content item, or having a page-id to content-id translation table. All of those options are used in practice in one CMS or another. Modern relational / SQL databases will be perfectly performant doing such queries occasionally. Indeed, while 11 hits on every page load might be onerous, doing 11 queries on every page edit (the worst possible case) is probably not going to be that big a deal. How often are edits done? Not very often, even in CMSs of thousands or tens of thousands of entries. If you're planning for massive (multi-million- or multi-billon-item) CMSs, we could talk about special indexing and optimization--but for even quite large production systems, 2 queries or even 11 queries per edit are not going to "break the bank."

And the reality is, even in CMSs that try to keep an in-the-database link between pages and content items, the links are usually incomplete. HTML links to external resources are so common, and so few user organizations are truly disciplined about keeping all content links fully described in-the-database. So even in those "worst cases," the number of in-the-database associated content items is less than you think.