text:
The information on this page comes from ML discussions, IRC discussions, in-person discussions at conferences and elsewhere, and from the issue tracker (both feature requests and bug reports). The idea here is to eventually come up with a concrete list of deliverables for the ZF2 MVC.

Please comment with additions, areas you think need clarifications, etc.

Controller layer:

Should provide access to headers

Should provide ability to both create and access headers

Should provide negotiation for complex values, including those with separators (e.g., Accept, Cache-Control)

Cache negotiation would be potentially interesting (e.g., for usage with ESI and Varnish)

Should allow retrieving cookie values

Should provide mechanisms for accessing superglobals ($_SERVER, $_ENV, $_GET, $_POST, etc.)

Potentially allow attaching filter(chain)s to superglobal types

Potentially allow specifying defaults for required keys

Should provide access to raw request body

Should provide auto-negotiation of JSON, XML in request body

Should transparently handle PUT

Should provide access to URI

URI should be encapsulated as an object

URI would provide access to path, host, schema, port, etc.

Interface and access should allow alternate implementations (e.g. Console vs. HTTP)

Potentially configurable ECGP style merging for HTTP env

Treat headers as metadata in non-HTTP versions

Flip side: allow building an entire request (for use in clients)

Should allow setting headers and header priority

Should allow setting cookie values

Should allow setting response body

Should allow retrieving headers, cookies, and response body (for use in clients)

Interface and access should allow alternate implementations (e.g. Console vs. HTTP)

Add EventManager integration

Fast/conditional matching

Easier, more flexible resource matching for RESTful requests

including resource relationships (e.g., /book/2/chapter/3)

Standardized console routing

Add EventManager integration

Support ZF1-style dispatching

Support Dispatchable interface

Potentially using direct mapping of controllers to classes

Potentially by pulling controllers/dispatchables from DI container

Should support short-circuiting – basically, as soon as a response is available, return it

This could be from dispatching, or from return values from events (e.g., a cache handler that returns full response, an ACL handler that returns "unauthorized", etc.)

Action controllers

Add EventManager integration to base controllers

Support ZF-1 style with helper broker, et. al.

potentially substituting plugins with EventManager

Support alternate style with real return values

Ideally return seeds view

Support RESTful styles natively

Server classes

Make dispatchable, so they may be attached to MVC directly

Must provide decorators for request/response objects

Refactor to use Zend\Reflection

Get rid of Zend\Server\Reflection, in favor of a PrototypeBuilder class

Move serialization of formats into Zend\Serializer?

Bootstrapping/Application context

Configuration driven?

If so, introduce configuration caching

Class-driven? (e.g., ZF1 bootstrap classes)

How and where should events be wired?

How and where are action helpers wired and injected? or are they?

What are "modules" vs. a "component model"?

Can this be addressed simply using namespaces?

View Layer:

What does an engine look like?

What pieces are common across implementations? (PHP-based, Smarty, Mustache, Twig, DTL, etc).

XHTML vs HTML?

Should we have multiple implementations? If so, which?

Many requests for design-centric implementation

Templates in separate project or tree from application

Templates edited by non-developers

How should the view layer integrate with the MVC layer?

Direct injection?

Via events?

e.g., event attached to Response::send() that introspects the "content" and metadata of the Response in order to prepare the rendered response

How should different contexts (XML, JSON, Atom, etc) be handled?

Where would official mobile integration live as well? (e.g., for selecting layout)

How will helpers work?

__call() is slow

But the best way to provide migration from ZF1

Strategy pattern may not be the best solution

But, again, best way to provide migration from ZF1

Some helpers require injection of non-view-related objects

Some helpers need knowledge of view configuration (e.g. XHTML? encoding?)

Should those be part of a view engine? or simply configuration?

Can helpers be used across engine implementations?

What about caching?

Support for edge side includes?

built-in caching for...

template tokens?

Model layer:

Common "model-like" infrastructure?

Collections (countable, iterable, ability to return items of specific class, etc.)

Serialization interfaces/classes (toArray, fromArray, toJson, fromJson, toXml, fromXml, etc.)

Interfaces, traits for "configurable", "eventful", etc.

De-couple view layer from Form classes

Move to view helpers (which would take elements and create HTML representations from them)

De-couple filtering and validation from elements

Move to brokers?

Move to form-level, to allow moving chains between domain models and forms (chains would be used to seed values and error messages)?

As changes in MVC strategies are implemented, tooling needs to support the changes.

Migration from ZF1

Must have some sort of "compatibility" layer

May 11, 2011

This is not a specific comment on the content, but about "But the best way to provide migration from ZF1". Now we have the chance to get lost of old habits and remove slow functions, so my advice is not to look to the ease of migration but rather the required final result.

May 11, 2011

It would be good to have the ability to have route based bootstrapping. This would allow for bootstrapping on certain route criteria. Modules themselves, if we go to a component module over an application module stance, would not need bootstrapping.

An additional option would be to have a merged config, pulling the main config followed by config files for the modules, all of which gets cached until the hash of the collective modules change.

Since there is no interrogation, autoloading or reflection of bootstrap classes this should in theory improve speed.

May 12, 2011

Dependency Injection can largely answer this, actually. In the prototype I've been working on, the router simply determines what controller to use, and you then pull the controller out of the DI container – which gets you only the dependencies needed for that request.

Of course, it requires that autoloading be setup correctly – but, again, new features such as the ClassMapAutoloader make that reasonably simple to accomplish.

I'm personally of the opinion that a module should be simply a namespace for code used by the application.

May 11, 2011

+1 to "doing it right" over keeping backwards compatibility

May 12, 2011
May 12, 2011

Indeed.

May 12, 2011

In some large Zend Framework projects i've found the default route-to-module+controller mapping of ZF1:

admin/registration/data-viewer --> modules/admin/controllers/RegistrationController.php

a little too restrictive. I've toyed with hacking a HMVC-alike controller/router setup into ZF1, but never did get very far. It would be nice if ZF2's MVC could be configured so that we could set up routes like these:

admin/registration --> modules/admin/controllers/RegistrationController.php

admin/registration/data-viewer --> modules/admin/controllers/Registration/DataViewerController.php

admin/registration/data-viewer/by-site/id/99 --> modules/admin/controllers/Registration/DataViewer/BySiteController.php

In my mind, this routing process doesn't need to be automatic like the current Zend_Controller is by default. All routes would be defined upfront by the application (ie: map route directly to name of controller class), as there is so much ambiguity possible in the URI->file mappings that complexity and low performance would become issues.

admin/registration/data-viewer/by-site/id/99 could possbily dispatch to one of:

modules/admin/controllers/RegistrationController.php (call dataViewerAction)

modules/admin/controllers/Registration/DataViewerController.php (call bySiteAction)

modules/admin/controllers/Registration/DataViewer/BySiteController.php (call idAction)

( + All of those again, but with indexAction instead of the action method noted)

Perhaps this could be implemented as a new router type coupled with storing controllers in a PSR-0-compatible way?

May 12, 2011

In the prototype I've been working on, I got rid of the concept of modules from routing, and instead did like you suggest: route maps to controller name. I then pull the controller from the DI container – which allows me to map controller names to actual classes.

Additionally, I treated the "application" directory as simply a location for code, and a "module" is simply a namespace within that directory. As such, the hierarchy can be arbitrarily deep.

Between these two concepts, I think many of your questions are answered.

May 13, 2011

Excellent! That's exactly what I was hoping for.

May 12, 2011

I'm looking forward to the EventManager integration, I looked at using Symfony's EventDispatcher lib with a ZF1 app I've been prototyping. While I got it to work, it feels kludgy.

In terms of module re-usability, my questions are:

Can modules provide js/css files for an application to load and potentially override?

For the view layer, it'd be nice if an application could override the templates bundled within a module without changing the module's template/view code.

May 12, 2011

You can go ahead and use the ZF2 EventManager today, if you want, if you're running on PHP 5.3; any feedback you have to give is welcome.

Regarding module re-usability, public assets are always a problem. While we can do things like symlink these into the public/ tree, that doesn't work on all operating systems. I'm hoping that perhaps we might integrate Assetic, as that approach looks quite thorough and promising.

What you discuss with the view layer is similar to the way the current script path works currently – but it can be tricky to get right. A more thorough solution would be to adopt HMVC principles. I think HMVC may be an interesting approach – but I'm not entirely sure if we should target that for 2.0, or a later revision (2.1 or beyond) due to the complexity involved.

May 13, 2011

The way I've handled static assets from modules in ZF 1 is to have a standalone 'static.php' script in public/ with some extra rewrite rules that point to it. The script then knows based on the URL which module and path to map the request to (e.g. /blog/images/foo.png might go to /application/modules/blog/public/images/foo.png), and it can create symlinks (where appropriate) to speed up future requests. I'd not seen Assetic though, looks interesting.

May 15, 2011

I think this static.php like solution is unnecessary. For example to create thumbnail images I create a route object in Bootstrap.php of image module. The URL is: image/thumb/something_30x30.jpg. First time the module catches the request, creates the image file and moves it into public/image/thumb directory. Next time the webserver will serve the previously generated static file. You can also use this idea to use module based javascript and css files.

May 30, 2011

+1 to the Assetic implementation. I have been playing around with it locally for a week or so and it makes asset management super easy.

May 12, 2011

I propose the use of Symfony2's MVC. Any requirements not met OOTB can be added via ZendMvcBundle.

May 13, 2011

I have to say, that i actually really like Symfony2's approach to MVC. It is clean and without magic. But the implementation seems to favor low code quality. There are public properties all over the place.

so -> +1 for similar approach and cleaner implementation

May 12, 2011

Hi two things,

*) +1 to "doing it right" over keeping backwards compatibility

*) I do not see any suggestions on how to tie everything together such as being done in

Zend_Controller_Front::dispatch(): creating request, response, dispatcher, router, firing plugin hooks, ...

I see the EventManager recurring a lot, so the hooks are being handled, but what about putting everything together? Will there still be a frontcontroller? If so what will be its role?

May 12, 2011

The front controller would still exist, but it would be much smaller – it would compose an EventManager and the router, and would handle dispatching directly. That leaves matching controller names to controller classes – which can be done by either composing a DI Container/Service Locator, or by composing something like a plugin broker. Overall, though, its role is greatly diminished; it no longer acts as a global registry.

May 12, 2011

My notes:

Would be nice for modules to be first class, they are rather tagged on in ZF1

Internally using DI sounds good, though we would need a good way of doing all the wiring

How would we provide default wiring?

Could we have wiring "profiles" for configuring different setups quickly? For example CLI.

With general wiring (events, di and helpers), I think we should try and keep it as simple as possible. Its one of the hardest things for new users to grasp and introduces a lot of magic IMHO. So whatever happens we are going to need clear docs.

Do we have need an extra documentation requirement that would list all events etc involved with a component, where they are configured and where they come from?

+1 to "doing it right" over keeping backwards compatibility

One usecase to consider would be having a mix of ZF1 & ZF2 code, if this is supported it make it easier to refactor an application over time.

Models, wouldn't go too deep into this, would be great to provide some basics (Collections etc) but DC2 support would probably be better.

+1 for decoupling the form components being able to grab the chains would be very helpful.

May 12, 2011

I've commented on modules above – but let me illustrate:

<ac:macro ac:name="code"><ac:plain-text-body><![CDATA[

The idea is that application/ is just a directory containing namespaced code. The tree follows PSR-0 guidelines. You define a module as simply a directory/namespace containing related code that belongs in the application. The hierarchy can be arbitrarily deep, and follow whatever conventions you need.

In the router, you're then aiming to identify a "controller" – matching a route should de facto indicate we know what the controller is. We then pull the controller object from the DI container, which allows us to match that controller name to a specific controller class – and we know how to locate the class due to the autoloader – which can be either a PSR-0-compliant path-based autoloader, or a ClassMap autoloader.

Basically, it makes the story of modules dirt simple. It also much more directly provides a path for reusable modules.

Regarding DI wiring, we have two goals: application performance, and developer performance. The former is largely solved currently. The latter is where wiring comes in. Ralph is working on some functionality for auto-generating wiring from reflection, and also some tools for implementing "first-run" compilation. You as a developer will still need to indicate things like aliases and explicit dependencies, but the work will be relatively simple.

Part of that work is also to provide per-environment definitions and application containers. This should aid situations like testing and re-purposing for the CLI.

Moving on to events – yes, we'll need to list all possible events in the documentation for each class, as well as the "aliases" each class might listen on. Most even wiring is actually very explicit: "I want to listen to these events on this class". The main problem is determining where we want that wiring to be made within the application definition. The bootstrap makes most sense, but we need to define exactly what that is.

May 14, 2011

I like the simplicity of this. Aesthetically, I prefer the old style of lower-case plural directory names, but I don't think they are worth the added loader complexity.

Where would layout and view templates go? Should there be a way for modules to participate in Bootstrap?

May 16, 2011

RE: lower case plural directory names: one issue I've seen is that developers are not aware that the classes in there are under a module namespace, nor where to find that namespace. While we have documentation around this, having the same rules apply to this code as general library code adds consistency, which aids in learning the framework.

RE: layout/view templates, asset management such as templates is something we need to discuss with relationship to the view layer.

The prototype I've been working on uses Mustache as the view implementation. As such, I'm putting all templates under a single hierarchy. This resolves one issue many have raised with ZF1 – they have separate design and development teams, and need to keep these as separate "projects" in version control. However, it raises new questions when it comes to reusable module development – where do the templates for a given module live?

So, let's begin discussing that. <ac:emoticon ac:name="smile" />

Third, regarding modules participating in the bootstrap: I see 4 key areas we need to address:

How do we inform the DI container about module classes, and how do we provide injection alternatives?

How do we route to controllers in a module?

How do we serve a module's public assets?

How do we extend or use templates within a module?

So, let's add these questions to the list. <ac:emoticon ac:name="smile" />

May 15, 2011

Would the controllers still support sub dirs?

With configuration/wiring I think it would be good to support various levels. By this I mean for simple apps you probably can have one Mega configuration place, but for complex apps you may want to split into smaller files either per module or something similar. A way of configuring this would be very useful.

As for the bundle idea, I can understand why people would want this but for me it would not help, so I would put it as a low priority. Having a way to make modules self contained would be good but I don't think perbuilt bundles for things like acl make that much sense as they would be a lot to maintain and make the framework over prescriptive IMHO. One of the things I love about zf is it lets you work in your own way, this flexibility is very important for enterprise development.

May 16, 2011

Controllers could appear anywhere in the application hierarchy; you simply need to:

route a particular URL to a unique controller "name", and

map a controller name to a specific class

Likely we'll provide some tools for automating this (either through conventions or via API annotations); the point remains, however, that a controller can live anywhere.

Regarding bundles/modules, the idea is not that the framework would maintain these, but instead provide an infrastructure for others to write and maintain them. As such, you, as a developer, can pick and choose what if any third party modules you want to incorporate, and also write your own re-usable modules for your own projects.

May 12, 2011

Some good stuff here. A few thoughts:

Bootstrapping/Application configuration -

Personally I prefer to do as much as possible of my application configuration in PHP (i.e. _init() methods in the bootstrap class), instead of through application.ini, I just think it's easier to see what's going on that way. Some of the things that show up in the application.ini are not especially intuitive (e.g. "resources.modules[] = "), and I think this can cause confusion for beginners. I'm not suggesting ZF2 should move everything into code, but I hope ZF2 doesn't go too far down the config file route.

Perhaps one way to improve things would be to have the default application.ini created by Zend_Tool to be a little more like the default php.ini files, containing various (all?) options in there by default (even if they are commented out), with comments explaining what they do.

On the topic of configuration, I'd also like to see some separation of application configuration and environment configuration. Imagine you have an application that runs in various locations and is worked on by a few different developers. Currently in ZF we have application-level things like the bootstrap class name, autoloader namespaces (things that will be the same in each instance of the application) in the same file as environment-level things such as database login details (which will likely be different for different developers). You'd often want the application-level stuff to be under source control whilst you keep the environment stuff out, this is a bit awkward to do currently.

Models - I like the idea of some interfaces to provide some structure around current conventions, e.g. a 'configurable' interface which presumably would define setOptions().

Other things -

I'd like to see support for controller namespacing, which I think is what Adam is suggesting above. The obvious use case for this is admin routes, which I think a lot of people use modules for. So you'd have /news routing to application/controllers/NewsController.php, and /admin/news routing to application/controllers/admin/NewsController.php. This shouldn't replace modules <ac:emoticon ac:name="warning" />, it just provides a way to group controllers together without having to use modules (overkill) or custom routes.

I hope there is some consideration towards reusable modules, which is something that pops up quite often. I wouldn't expect this to be in ZF 2.0, but it will affect some of the things being discussed here so I hope it's at least on the radar.

May 15, 2011

I agree, we should start moulding modules into self contained units and move other parts to a better suited location.

It would be good to drop the automatic determination of anything past the base route (not including modules).

By this I mean primarily the only route should be /controller/action. Any other extra, like an admin route for example, would be defined by the user in the routes config.

May 16, 2011

I agree here. In the prototyping I've been doing, I've actually defined all of my routes explicitly, as this makes it really easy to determine exactly what class and method is being invoked. That said, I think having a conventions-based route such as "/:controller/:action" makes a lot of sense as well, as does a "RESTful" route. Beyond that, explicit should rule the day.

May 21, 2011

In regards to modules, how would these be "Bootstrapped" ?? I mean will the system scan the folders and files as in ZF1 or will there be some "cleaner" way?..

could a scan of everything on every load be done away with?

May 18, 2011

Somewhat linked to making things easier to learn, any chance we could move away from INI files? A simple native PHP configuration format could remove some of the pain from array handling (repetitive long worded prefixing is not that simple).

May 14, 2011

How should different contexts (XML, JSON, Atom, etc) be handled?

Where would official mobile integration live as well? (e.g., for selecting layout)

Personally, I'm a big fan of the context switch action helper. It minimizes controller action code duplication. Potentially we could add a 'mobile' context. I see no reasons why contexts could not apply to layouts as well.

May 15, 2011

I already made up something like that. It is important to couple a context switch action helper to a frontcontroller plugin, because for mobile views it's also required to switch a layout. More information here.

May 16, 2011

ContextSwitch is nice, but it requires a lot of coupling:

It has to be enabled across all controllers, and...

has to know about the ViewRenderer instance and modify its state (to change the template suffix)

Additionally, if code cannot work with a certain context, you then have the controller needing to know how to deal with that situation (i.e., re-enabling layouts, re-setting the template suffix, etc.).

While I think in most cases, ContextSwitch works, I think this can be done in more transparent and easier to trace ways if we have it as part of rendering. A rendering engine would be told the current context (or derive it from the Request object), and then choose which renderer and/or template to use. And rendering can now potentially happen at the last possible moment – which means we know exactly what context we're in and whether or not to use a layout.

May 15, 2011

Some opinions on the controller layer...

I'll put my considerations on the rest in a different comment to make it not too long.

Please dont quote this comment as a whole

Please feel free to discuss also on IRC (#zftalk.dev, freenode) or on the mailinglists. As long as conclusions of any such discussions find their way back to this page.

Should allow setting headers and header priority

I'd propose to not do that. This would mean you need to do extra loops and possibly resorting. I'd propose to add the ability to define a callback that actually performs the ordering. We could then supply a callback ourselves that just sorts on priority, but do not enable it by default. This does imply that headers should be given the opportunity to contain additional metadata that can be used for ordering (and modification of those headers if someone would want to).

Routing Should imho not be related to MVC, proposing to move this to its own proposal/discussion. Rest: should be way better defined, and again has no place in this proposal imho. (Matthew, do you still have the pastie I gave you a /long/ time ago on how I'd do rest routing?).

Dispatching

Support ZF1-style dispatching

 < What is hereby understood, what parts are intended to be replicated. Does it make sense? Adding compatibility layers should not be an upfront requirement imho because it only limits the innovativity that's put in the new stuff. Compatibility layers can always be implemented in a later phase/stage.

Should support short-circuiting – basically, as soon as a response is available, return it

 Isn't that a characteristic of the Eventmanager?

Action controllers

Support ZF-1 style with helper broker, et. al. potentially substituting plugins with EventManager

 again, imho BC should not be a goal or consideration when designing new versions. Compatibility can always be dealt with in a later stage.

Server classes:

I agree they should be dispatchable, but not sure they should have decorators. An alternative approach may be to have it just define its own output, and skip the view layer alltogether.

Refactor to use Zend\Reflection

Get rid of Zend\Server\Reflection, in favor of a PrototypeBuilder class

Which one is exactly meant here? I'd propose to use a prototypebuilder class indeed which may then rely on reflection. Have this class work on-the-fly in development environments.

Bootstrapping/Application context

Should be a totally separate discussion, I'd propose mailinglist.

Modules

Imho only relevant to bootstrapping (will also address this under views).

(edit, added: )

Tying it all together

In the comments I'm reading that the idea is to have a frontcontroller. What would be the position of such thing when we also have a DI tool. Wouldn't it be a good idea to store it all in one common DI framework and just rm Zend/Controller/Front.php?

Also, it happens to me a lot that an 'action' (or its missing presence) is represented only by a view and a few mappers. If one could also define actions by simply putting them in your routes config or as key of a property of the controller class we would not have to define empty methods for these, and because of the fact that they're missing skip a few things in the dispatching process and potentially improve performance there.

May 16, 2011

Responding to some selected items only here.

Should allow setting headers and header priority

I'd propose to not do that. This would mean you need to do extra loops and

possibly resorting. I'd propose to add the ability to define a callback that

actually performs the ordering. We could then supply a callback ourselves that

just sorts on priority, but do not enable it by default. This does imply that

headers should be given the opportunity to contain additional metadata that can

be used for ordering (and modification of those headers if someone would want

Agreed; this approach makes a lot of sense, and also goes along with the general

idea of request/response objects having both metadata and content.

Should imho not be rlated to MVC

Why not? Routing is how we analyze the Request object and match it to a

dispatchable; I'd consider this fairly central to the MVC. I think the actual

implementation of routing should be a separate proposal and discussion, but

where and when routing happens in a request lifecycle is definitely a part

of this discussion.

Should support short-circuiting – basically, as soon as a response is

available, return it

Isn't that a characteristic of the Eventmanager?

Yes and no. It is in that you can short-circuit an event at any time. However,

whatever triggered the event then needs to determine if a short-circuit

happened, and act on it. When I say we should short-circuit as soon as a

response is detected, this is the role of the front controller – identifying if

an event has resulted in a response, and, if so, returning it.

Server classes

I agree they should be dispatchable, but not sure they should have decorators. An alternative approach may be to have it just define its own output, and skip the view layer alltogether.

That's what they do right now. However, to be compatible with the MVC system,

they would need to return a response. The response itself doesn't really require

decoration, but decoration may be needed on the Request to ensure that the

server can get the information it needs out of it (e.g., pulling XML or JSON

from the raw request body and parsing it into parameters).

Refactor to use Zend\Reflection

Get rid of Zend\Server\Reflection, in favor of a PrototypeBuilder class

Which one is exactly meant here? I'd propose to use a prototypebuilder class indeed which may then rely on reflection. Have this class work on-the-fly in development environments.

What I was getting at here is that there is a fair bit of code duplication

between Zend_Server_Reflection and Zend_Reflection. Zend_Server_Reflection's

main value-add is being able to provide method prototypes. My suggestion is

creating a Zend_Server_PrototypeBuilder class that consumes Zend_Reflection in

order to create the prototypes.

Bootstrapping/Application context

Should be a totally separate discussion, I'd propose mailinglist.

That was the problem with ZF1 – we talked about bootstrapping and application

context after we developed the MVC, and they were literally tacked on. This

should be an integral part of the MVC discussion – it helps inform a number of

decisions, ranging from configuration to caching to how developers setup the

application initially. I'd rather not go through the 0.6.0 - 0.9.0 cycles again.

<ac:emoticon ac:name="smile" />

Tying it all together

In the comments I'm reading that the idea is to have a frontcontroller. What

would be the position of such thing when we also have a DI tool. Wouldn't it be

a good idea to store it all in one common DI framework and just rm

Something somewhere along the line needs to:

map the route to a controller name

retrieve a controller based on the name matched

dispatch that controller

handle any errors that may occur in the above steps

That's what the front controller is for. DI can be used to handle step 2, but is

hardly a solution for the rest of it, other than to provide objects as needed.

May 15, 2011

Some things on the view & model layer (and modules):

What pieces are common across implementations? (PHP-based, Smarty, Mustache, Twig, DTL, etc).

The common denominator here would be:

Variables assigned to layout(/view)

response (the body) from layout(/view)

The implementation of each solution (assuming a webenvironment) should also return a content type and encoding. Wehther it retrieves those from the actual view/layout invoked or whether this is determined by the implementation should be left up by the implementation.

Should we have multiple implementations? If so, which?

Obviously we should ship a php implementation. If the community is able to supply other implementations as well I think we could just ship these. They however should imho not slow down development of the fw. If we choose to ship a smarty implementation by default I would propose to ignore Smarty <V3, designers can also learn new things.

As discussed on IRC there should be a paragraph or chapter in the manual describing the considerations for choosing an implementation (like: if you work with programmers, there's no reason to separate html from php). I will do this - when the time is there.

How should the view layer integrate with the MVC layer?

Direct injection?

Via events?

Why not both? One could invoke them through events, locate them through DI. I'm not too much into the DI stuff, but that would also provide a solution for:

Templates in separate project or tree from application

Templates edited by non-developers

If you could tell the DI how to map the views, you could specify a totally different location for views than the rest of the app.

Servers & contexts

How should different contexts (XML, JSON, Atom, etc) be handled?

Isn't that inherent to how the server classes are dealt with? As mentioned in my other comment, I would not rely on views for those. If we do, how do we keep into account versioning of api-servers?

Where would official mobile integration live as well? (e.g., for selecting layout)

The context detection should imho find place in the request object (if this costs performance make it optional). Then each view-implementation (php, smarty, twig) can choose for itself how to deal with these. I think the defining of callbacks here would be best since I suppose each user will have its own way of doing this. This again could also be tied in how the DI does its location of these dependencies.

How will helpers work?

Again, imho BC should not be a consideration here. There's always a way to think of a compatibility layer since by the end of the day ZF1 will do the same as ZF2 (that is: primarily serving websites). One should also be able to supply mere callbacks (closures in specific) that act as, and can be invoked as such, viewhelper.

Cant we build some buildtool here as well that simply detects all available viewhelpers, and create a class that has one property and method per helper, the property containing an instance of the respective viewhelper which is lazy loaded. And the method that lazyloads the viewhelper and proxies to it when called. This strategy would allow a __callback fallback for BC reasons as well as development environments.

Some helpers require injection of non-view-related objects

Disagree here. If a viewhelper needs objects, apparently they're related to the view and should be passed on to the view. If you pass on the view object to the helper they'll also be available to the helper. Alternatively we could choose to go the lazy way, and simply inject the DI tool (which we may do so anyways), and simply retrieve the required objects from there.

Some helpers need knowledge of view configuration (e.g. XHTML? encoding?)

Should those be part of a view engine? or simply configuration?

Configuration can be retrieved via the DI tool (see above), stuff like encodings can (should) be retrieved from the respective implementation (php, twig, smarty). Where the implementation get the requested values from is up to the implementation.

Can helpers be used across engine implementations?

Why not? I'd say that each engine implementation can have its viewhelpers, and possibly fall back to the default php ones.

Caching

Not my area of expertise. I'd say it's up to each engine implementation what should be cachable.

Forms

FORMS HAVE NO PLACE IN MODELS (there, said it. May blog on it). What may actually be useful is an interface that can be validated against so that the controller can simply call the model layer with form values to verify their validity. Perhaps a Zend\Validate\InputCollection may be appropriate here which can then be extended by entities that are part of the modellayer which both mappers as well as controllers can check against.

Move to view helpers (which would take elements and create HTML representations from them)

Should viewhelpers actually be the default? I have a lot of forms that are highly customized, and as a result I just write plain html (and sometimes invoke their renderes), like:

<ac:macro ac:name="code"><ac:plain-text-body><![CDATA[
class myform extends form {
public function render()Unknown macro: { returns "<form><input name="foo" id="foo" />".$this->elementBar."</form>"; }

I'm not saying this is the most neat way, but I can imagine lots of people finding it handy when it comes to the somewhat 'more complex' forms. We should take mmvc in consideration here.

modules

I'm not sure what I was planning on saying on modules. Except for that that really should be a separate discussion together with bootstrapping. They should however be self-contained containers (which may be reliant on other modules). Therefore I would also put all the public stuff (icons, js, css, you name it) inside the module dir. This could be done by e.g. using a rewriterule that maps everything for ^media/$1/$2 to APPLICATION_PATH.'modules/$1/public/$2 . That approach would be a cross OS and cross FS solution (opposed to symlinks, hardlinks, etc), and I think most webservers support such a solution.

I'd also like to use this opportunity to propose (once again) to change the name of the module 'default' to 'core'. This fits way better in the theory of modules being selfcontained containers (still wonder if that's a pleonasm - or not).

layouts

I assume we'll treat layouts no differently than views, and simply remove zend\layout and merge its functionality of that of zend\view?

Migration from ZF1

Again (feel like I'm repeating myself here. Wait... I probably am), once we have something that works the way we want it, and essentially still does the same as ZF1 does (serving websites for the biggest part), a compatibility layer will always be possible. We should not be held back for BC-reasons only (we've been doing that ever since June 2007 now).

May 16, 2011

The common denominator here would be:

Variables assigned to layout(/view)

response (the body) from layout(/view)

*

The implementation of each solution (assuming a webenvironment) should also return a content type and encoding. Wehther it retrieves those from the actual view/layout invoked or whether this is determined by the implementation should be left up by the implementation.

I disagree about your first point. The reason is that I think in many cases, a

view implementation doesn't need variables directly assigned to it, but rather

needs variables or an array/object passed at render-time. Even with layout

variables, these can be part of a Layout Model object (or array).

I agree about encoding and doctype being something that can be left up to the

implementation; however, still, somewhere it needs to be defined, and I've yet

to find an implementation of a solid rendering agent that was not aware of at

least encoding. Additionally, most helper systems we might create will need to

know encoding and doctype in order to render HTML markup correctly – so these

values need to be managed in such a way that they may be injected.

How should the view layer integrate with the MVC layer?

Direct injection?

Via events?

Why not both?

You're absolutely right, and any given project could make effective use of

both. It doesn't need to be either/or.

How should different contexts (XML, JSON, Atom, etc) be handled?

Isn't that inherent to how the server classes are dealt with?

It's not the same discussion at all. XML, JSON, and Atom can all be used outside

of the server contexts.

how do we keep into account versioning of api-servers?

That one's easy: via discrete routes.

The context detection should imho find place in the request object (if this

costs performance make it optional). Then each view-implementation (php, smarty,

twig) can choose for itself how to deal with these.

I hear a requirement for the Renderers on the Request object in this statement.

<ac:emoticon ac:name="smile" />

...

Cant we build some buildtool here as well that simply detects all available viewhelpers, and create a class that has one property and method per helper, the property containing an instance of the respective viewhelper which is lazy loaded. And the method that lazyloads the viewhelper and proxies to it when called. This strategy would allow a __callback fallback for BC reasons as well as development environments.

IMO, helpers should be objects, and allow developers to call any method. As

such, the above makes no sense – in most cases, I'd be retrieving a helper

object and calling methods on it:

<ac:macro ac:name="code"><ac:plain-text-body><![CDATA[
echo $this->broker('url')->simple('action', 'controller');

As such, the one property/method per helper makes little sense. (The evolution

of ZF1 has shown that in more cases than not, helpers become stateful.)

Some helpers require injection of non-view-related objects

Disagree here. If a viewhelper needs objects, apparently they're related to the view and should be passed on to the view. If you pass on the view object to the helper they'll also be available to the helper. Alternatively we could choose to go the lazy way, and simply inject the DI tool (which we may do so anyways), and simply retrieve the required objects from there.

I think you misunderstood my point. Consider the case of the URL view helper; it

needs access to the router and the request object. The View object likely does

not knowledge of these, but the helper does. We shouldn't use the view

object as a general registry.

...

Should viewhelpers actually be the default? I have a lot of forms that are highly customized, and as a result I just write plain html

This should always be possible. I'm simply arguing that we should have view

helpers that can generate standard input elements and/or labels based on the

form element provided. You'd always be able to simply grab metadata out of it

and/or hardcode your forms however you see fit, so long as the names used match

those you're validating.

May 15, 2011

I am definitely with Dolf Schimmel (Freeaqingme) on the case in point of BC. ZF2 should allow us to do the same work as before, but in a much better way. We should get ZF2 running in the best way, then add a compatibility layer later for those that want and/or need it, otherwise we will be left with a half way house that doesn't quite meet expectations.

May 15, 2011

1. Caching over HTTP-Protocol (Browser, Proxy)

is the response of this request cacheable (GET, non secure, non authenticated, Cache-Control, ...)

was the response already cached and is fresh (If-Modified-Since, If-None-Match, ...)

enable/disable caching (Cache-Control, Expires, Last-Modified, ETag)

send a cached response (304 Not Modified)

2. Caching complete output (ZF1: Zend_Cache_Frontend_Page)

generate a unique cache id by request and automatically send the response if it was cached before

else write response to cache on send

This could also be used to automatic handle HTTP caches

and to create a simple (not real) gateway cache

3. Caching complete output using a static file (ZF1: Zend_Cache_Frontend_Capture + Zend_Cache_Backend_Static)

configure httpd to call a php script if requested file doesn't exist

create the requested file

-> Need a way to clear old files

4. Caching parts of output (ZF1: Zend_Cache_Frontend_Output)

This could be used to cache results of view scripts (e.g. partials)

5. Edge Side Includes (ESI)

caching only parts of the response by http client (in most cases a gateway)

replace cacheable parts with a esi placeholder including an URL where to find this part as own response

this could be implemented like a partial:

<ac:macro ac:name="code"><ac:plain-text-body><![CDATA[
echo $this->esi('partials/cacheable.phtml', array('var1' => 'val1', ...));
// <esi:include src="http://..." />

the method should create a unique URL which can be requested to output the result of this partial and output it as esi tag

It would be very great if we don#t need (but can) create an own action for this

How to support variables other than strings ?

http://symfony.com/doc/2.0/book/http_cache.html

http://www.mnot.net/cache_docs/#WORK

http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html

http://www.w3.org/TR/esi-lang

May 16, 2011

Regarding caching over the HTTP protocol, I agree that it's primarily a function of the response. The biggest question is if we can make this as easy as possible to implement, with support for all the available headers we could provide.

Regarding ESI, yes, you're absolutely right – it can be as simple as a partial, and if the partial can have access to a URL-generation helper, this is trivial to accomplish. It should, however, hook into the HTTP caching as well.

Jun 04, 2011

Methods sf2 already ships with to handle caching headers is a good point to start here.

Also for people not running an own gateway/proxy it could be very interesting have one on application side

similar to the page cache but by follow the HTTP specification. (sf2 reverse proxy)

I'm not sure this would be part of the MHV or the cache component - it needs both.

I don't mean to copy all the abilities from sf2 but there are already good points solved with a simple API.

May 16, 2011
<ac:macro ac:name="unmigrated-wiki-markup"><ac:plain-text-body><![CDATA[Ok, ZF2 MVC.
Most of what Matthew has said re DI / Eventing is ok IF it performs as well as the old style system. I have some fairly serious concerns about the performance of a PHP system where every object is injectable. Dependence on reflection API is similarly concerning regarding performance.
Next, here's my requirements wishlist for ZF2... some address MVC specifically some are more broad... but this seems like the right time.
1. Phar. Phar should take a front and center stage in ZF2. We should be able to package ZF2 as a .phar, package entire apps as phar and package modules/add-on libraries individually as phars.
2. A 'component' as I define it would be a lot larger than today's modules. It should allow you to tie into all parts of the MVC system, from bootstrap (route injection, db dependency additions) to views/controllers/models/forms/etc as well as providing application resources like static html/css/js/etc.
3. Components should be able to override the app they are installed against. For example if I install Matthews_Awesome_Blog component... I should be able also install Kevins_Wicked_Theme component, which will override. I'd suggest a priority list (think z-index) where components marked with a higher priority override. I would suggest developers ship components with a standard priority, but that it should be configurable by the user through Zend_Tool.
4. ZF2 itself should take a version-based approach to installation. We need to follow Microsoft in their concept of a versioned GAC. There seems to be a move to a pyrus deployment system, and while it supports versioning, I still see the default approach being one of updating a single central library or including ZF in the app itself. Our use case demands that each and every ZF update go through full beta testing, and this takes a variable amount of time per site... so the current workflow is to download and install the new ZF version to /usr/share/php/ZendFramework/ZendFramework-1.11.6 ... so it is available to all sites, and then the developers can bump the version dependency as the sites are sent through QA control again. Updating a live ZF library in-place must be deprecated. This technique is fully explained in my book in the Zend Framework chapters.
5. Overhaul the caching mechanism integration. Caching should be controlled as part of the design, not bolted on. I suggest that routes take a caching parameter, and that Zend_Registry and Zend_Db also have a caching-first design. If your entire page is cacheable, you should be able to mark it as such. I would suggest that an ICacheable advisory interface for controllers, models, etc could be added, and that methods like somethingCachedAction($cache) {
$cache->cacheFor = 3600;
//action as normal
could be detected by their name... I think we really need to take an architectural approach to caching, and ensuring its a default-on feature for auto-generated boot-straps.
6. Layout, partials, actionstack... need to re-think the ZF1 design... its totally inefficient, but the motivation is right. A good 80/20 use case would be a layout item that says how many items are in a shopping cart. Right now, we can either use an action-stack, or simply include the code directly in the layout – which is fugly, or override Zend_Controller_Action and add a custom site-wide init() method to include that code (also fugly)... another good use case would be showing the logged in username and role (presuming a db/model task to determine and not a simple session variable/zend_auth:getIdentity read directly in the layout).
7. PRG... Post-Redirect-Get... should become the default for post forms in ZF2. This means integrating Zend_Form, controllers, the FlashMessenger, and routing into one elegant solution. Much to argue about over best way to do this, but one simple requirement that pressing 'back' in the browser never prompts for a 'do you want to resend this information' dialog should suffice here.
8. Review the translation overhead. Mutli-lingual operation is awesome, but I wonder if a custom build system might be able to create an internationalization compatible version vs a non-compatible version. The latter being presumably more performant.
9. Security-By-Design. Need components for privacy compliance and security. Need major improvements to the Zend_Version system and to the site-administrator notification systems. These shouldn't be bolt-on, but default components. When Zend releases a User-Security-Notification, every defaults-installed ZF2 app should start whining to the system administrator that an upgrade is needed. I'd suggest a probability based system that triggers on 1% or so of hits, and can track what notifications have already been sent... set the system to send an email at least every 24hr's until the site is patched. This is critical to maintaining the frameworks reputation for security by policing the installed base. Privacy-by-design means creating a data tainting system ... and similarly SQL issues can also be controlled through data tainting. A system that addresses where data came from and where it is safe to display/use would be hugely helpful towards privacy and security. We should get rid of the current escapeFilter approach in support of something more sophisticated.
10. The ZF2 code should be simple, and easy to learn. If the early-adopter must learn concepts like FSM and DI to create a secure, performing application – then we've failed, and will end up like Java for web development.
11. Improve filter suite to include a number of best-practices filters. For example filters for a phone number, email, addresses, credit cards, etc... we have some basic filters, but we should strive to provide a full every-day-usage set.
12. Tighter integration of Auth/ACL... tie it closer to the routing system, and to the controller layer.
13. Concerns regarding ORM integration. Is doctrine still going forward as the Zend_Db replacement? Are we going to better map to PDO (specifically, reworking the _quoteIdentifier mess, using proper prepared queries, array binding, etc)
14. I'd like to see an RIA-specific use-case outlined for use with mobile frameworks (phonegap, jqtouch, etc)... and for RIA app development for ajax-heavy apps like those that use dojo, extjs, etc. Can we short-circuit most of the MVC in these cases... maybe a Zend_Controller_LightweightAction or similar which wont initialize the view systems.
15. Better integration for API's.... Zend_Service is cool, but each component is really different, and there's no way to handle the security-policy. Eg does each properly check SSL certificates, verify peers, etc. Can we maybe start from common objects... Zend_Service_Rest... Zend_Service_XMLRPC... Zend_Service_SOAP... Zend_Service_Post... as base classes for the specific implementations?
16. Better HTML sanitization component. Maybe look at bringing in a third-party lib that does this well?
17. CLI scripting use cases. Standardize environment variables for framework path discovery. CLI bootstrap, CLI arg processing, gettext, etc..
18. Web-Cron subsystem, engage a script on probability, and interval checking. Important for some deployment scenarios.
19. Fix Zend_Session design issues. Make setExpirationSeconds work properly.... see point 18... integrate clean-up tasks. See ZF-4753 which is still outstanding.
20. Address contribution/process overhead. See http://radar.oreilly.com/2011/05/process-kills-developer-passion.html
There's more, but that'll get the ball rolling.
– Kevin
May 16, 2011

Addressing one thing at a time...

Most of what Matthew has said re DI / Eventing is ok IF it performs as well as the old style system. I have some fairly serious concerns about the performance of a PHP system where every object is injectable. Dependence on reflection API is similarly concerning regarding performance.

I think you'll find it performs better than ZF1 – in large part because only the objects required for a given request execution path are ever instantiated.

Regarding your query about a system "where every object is injectable", I really don't get what your concern is. Objects are already injectable – the controller is injected with the request and response objects as well as the front controller parameters, and often injected with further objects by the various action heleprs in play (case in point: the view object is injected by the ViewRenderer). Injection has nothing to do with performance, but everything to do with how objects interact and the specific responsibilities of each object.

Is your concern that objects will always need to be pulled from the DI container, and what the performance impact of that is? If so, just remember this: you only get out of the container what you've defined in the container, and if you ever don't want to use it, simply use "new <Object>".

Finally, regarding "dependence on reflection API," the idea is that reflection will only ever be a development-time issue. It would be cached on a first-run basis during development, and tools will be created to generate code based on reflection for production purposes. These approaches can eliminate the Reflection overhead completely in production.

May 16, 2011

I'm referring to the extra calls to find a responsible object... for example we have Zend_Controller_Front::getInstance()... as i understand the new DI system it will be... DIContainer::getInstance('frontController') or something similar... which will then have to use a lookup in user-space to find which class is instantiated and tied to frontController. Not a big distinction, but * 100+ components in the system, it could end up adding a significant overhead...

Same goes for reflection... yes, its a development time concern, but it's also a runtime concern as the same extra lookup steps will be required... I'm guessing Zend Server cache backends and/or memcached are going to quickly become really important...

To be clear, I'm not against this approach, it just needs to be thoroughly benchmarked vs ZF1 performance and evaluated on a cost (runtime speed) vs benefit (testing + <10% use case) basis.

May 16, 2011

You're not understanding DI well. With DI, you define explicit dependencies and relationships. If I request the controller referred to as "entry-controller", I get that controller, and any dependencies it requires (and any those require, and so on down the chain). As a specific example, in the sandbox I developed, a given request to the entry controller retrieved the "EntryService", which composed a DataMapper, which composed a DataAccess object. (In my case, the DataAccess object was a MongoCollection, which meant I also needed a MongoDB and a Mongo object.) But that was it – I was only grabbing those. Those were dependencies. I created other objects over the course of the request: the mapper would create collection or entry objects, and a given request would create a "view model" object based on the request and execution. But the DI container only mapped the controller and the direct, required dependencies in order to run – and these were in the range of a handful to a couple dozen at most, all of which were used over the course of the request.

Also, I'm not grabbing the front controller from the DI container; I'm composing the DI container into my front controller, and the FC then pulls out a controller from the DI container. This ensures I only get the components necessary for that request. So, be assured, we're thinking of how to optimize execution paths – we don't want a situation like we have today with Zend_Application where any possible resource that might be requested across the application lifetime is configured for each and every request. We're aiming to only grab those we need for this request.

Now, as for Reflection, again, you're missing the point. By the time we get to production, it's no longer a runtime concern; we've compiled the Reflection information either into cached configuration or into actual classes. There are no longer lookup steps – you grab an object from the container, and it grabs what it needs; there's no lookup, as the code has been generated for that specific path. (In the DI prototypes, Reflection is used in one of two places: to instantiate a class with > 3 arguments, or to generate configuration for DI definitions. The former becomes a moot issue when you generate a ServiceLocator from the DI instance, as it then is capable of generating the entire instantiation argument string.)

I need to cleanup my benchmarking script for public consumption, but what I measured was running significantly (as in > 10%, and more on a factor of magnitudes) faster than equivalent code in modern versions of ZF1 that use Zend_Application.

May 16, 2011

I'm pretty sure I am understanding it fine, but we're talking two different things... you're concerned about loading unnecessary resources (a totally separate problem)

I'm talking about the overhead of the DI responsibility registry itself. There will presumably be thousands of calls to it over the course of a complex page request.

Each lookup to the container to find the responsible object is a lookup – and its called over and over again by different components...

Same for the reflection stuff, yes they'll be cached configs/classes or whatever... we'll still need to load those configs and there's still a hash-table search / mapping operation for each call. Overhead exists – the question is it significant.

The alternative is of course not to not-address the extra-loading of classes, but rather should it just be Zend_Controller_Front::getInstance() instead of DIContainer::getInstance('frontController') ... the latter requiring many extra userland PHP calls that the former doesn't. (find the object in hash table, return registered instance, etc). This is the overhead I refer to.

May 17, 2011

Please, before you make these assumptions, look at the work done in https://github.com/ralphschindler/zf2/tree/feature%2Fdi – most of what you think might be happening won't. When you generate a service locator class from the DI instance, there is no hash table, and few if any lookups happening.

Also, the suggested use case is that there is a single explicit call to grab an object from it, which grabs the controller and all its required dependencies. While I can certainly see a case for dozens of dependencies, the liklihood of thousands at once is slim – and a definite code smell calling for refactoring for that particular application.

That said, I'll get some benchmarking references up in the next week or so to provide more concrete information for you.

May 17, 2011

I've looked at the source... ServiceLocator::services and ServiceLocator::map are the two hash tables that will receive thousands of lookups. If every object access goes through DI with a context change, yes, this will get called repeatedly and yes you will have to do a lookup. I see no way around that.

How do you suppose ServiceLocator::get will not be repeatedly called?

May 19, 2011

Look at the prototype in git://mwop.net/zf2sandbox.git on the "feature/mobile-layout" branch. In there, I generate the AppContext class from the DI definitions. When I call, say "get('Blog\Controller\Entry')" on that class, it calls explicit getters in that class to get the various dependencies. It's not doing repeated lookups – it looks things up once per dependency. Yes, there are function calls to do so; these take fractions of milliseconds apiece, and in their aggregate will not lead to undue performance overhead.

I think your argument is that we'll be consulting the container many times over the request lifetime. But the prototype I'm working with is explicitly not doing this. It's instead using the following algorithm:

Instantiate the AppContext

Instantiate the front controller and pass the AppContext to it

Instantiate the router and route

Pull the matched controller name from the AppContext

Dispatch the controller

Return a response

The container is queried once for the controller, which then pulls in the dependencies needed for that controller only. This is a small, finite set of objects. I still instantiate a number of objects by hand – my service layer will create entities and collections, my controller will identify a view model object and instantiate it, etc. The point is that I pull only the required dependencies for this request from the container; all other objects are created on an ad-hoc basis and do not require the usage of the container.

I totally understand your concern. I had the same concerns originally – and as such, I developed the core of the container to be fast, but furthermore, created tools to generate explicit service locators so as to keep object overhead and lookups low in production. Additionally, if you look at the algorithm above, I looked for patterns that would limit the number of times the locator would need to be consulted.

May 19, 2011

I still think you can have those parameter-passing improvements without generic DI.

But, if you think you have factored out the overhead, I'll believe, and as I originally said 'sounds good if it can perform'... my worry enters in the controller/view/model/db layers as much as it does in the routing/dispatching/etc process.

If you are proposing non-registry patterns for calls like getFrontController, then it could be ok, however, if such a call needs to repeatedly hit the registry then there will likely be issues. If every class has to store class-level references to all the objects it works with by the means of a protected member variable, then I posit there might be a large amount of references when compared to a singleton approach.

I'm not against this model... it just has to perform, and I'm very concerned it wont.

May 16, 2011

10. The ZF2 code should be simple, and easy to learn. If the early-adopter must learn concepts like FSM and DI to create a secure, performing application – then we've failed, and will end up like Java for web development.

A few things: first, the more recent prototypes we've been playing with are not using finite state machines, so this is a non-issue. As for DI, we're currently working on developer tools to make DI incredibly simple to implement within your applications – tools that will scan your codebase and create definitions for you, and make your job simply indicating which specific implementations to use.

But your real point is about the learnability of the system. This is indeed a goal, but there will always be something the new developer will need to learn in order to use the framework. It will never be as easy as simply typing some code into Notepad and pushing it up on the web server; that's PHP's job. The job of a framework is to provide standards, conventions, and tools around development – and just like a hammer, you have to learn how to use those tools effectively.

Case in point: in this same comment, you mention developing with PRG by default, tracking versions by component, tighter auth/acl integrations, RIA, etc. Any of these may come into play in a given application, but might not. Should we leave them out as making the framework harder to learn? or do we provide good documentation so that the developer understands what they're for and when to use them?

What I'm getting at is that we can't single out any given concept as "too complex for the new developer." We need to look at what bare minimum knowledge is necessary to get up and running, and how to communicate this effectively. Additionally, we need to make sure we have good documentation to incrementally add to that knowledge so that new developers can expand on what they know in order to build better applications past the quick start.

May 16, 2011

Yea, I totally agree... if we do PRG, and the rest, we need to make sure the design is elegant so that its dead simple to learn... I think the DI could be the same, but the learning curve is steep. Java got web development really wrong, but they did it for all the right reasons. I'm also concerned about railsification or something similar to the nightmare that is DOJO build layers.... yes, you can make it work, but its massively complicated for a newbie to figure out. Learning lots of Zend_Tool commands or how to configure an xml config isn't the right direction either...

So for a requirement...

– Code level ease-of-learning should be maximized.

Ideas for this might include having off-the-shelf action controllers for common actions (much like Zend_Tool offers for errorcontroller) but maybe as a base class in the framework – but that goes back to the component model too.

May 17, 2011

I agree that code level ease of learning is important. Aside from some default impl's, keeping the API as flat as possible would also be good, i.e. not having to code through more than 1-2 class levels to use some functionality. This may necessitate having helper functions defined on controllers by default instead of using depth like $this->getSomeBroker()>getSomeHelper('cache)>cache('something') vs $this->cache('something'). Flatten it out for common use scenarios. DI, caching, and a few others spring to mind... There's a bit of a conflict in terms of explicitness vs magic which may work against some other dearly held OOP principles like ensuring the API reflects the class role. If controllers need to integrate DI, then the methods should be on the controller class - not buried three levels deep in the class deps.

May 16, 2011

4. ZF2 itself should take a version-based approach to installation. We need to follow Microsoft in their concept of a versioned GAC. There seems to be a move to a pyrus deployment system, and while it supports versioning, I still see the default approach being one of updating a single central library or including ZF in the app itself. Our use case demands that each and every ZF update go through full beta testing, and this takes a variable amount of time per site... so the current workflow is to download and install the new ZF version to /usr/share/php/ZendFramework/ZendFramework-1.11.6 ... so it is available to all sites, and then the developers can bump the version dependency as the sites are sent through QA control again. Updating a live ZF library in-place must be deprecated. This technique is fully explained in my book in the Zend Framework chapters.

First, even though we're discussing using Pyrus, the idea is that the ZF2 library will be versioned as a whole. If you install Zend\EventManager for v2.1.2, it will ensure that any dependencies are also of v2.1.2. We have no desire to maintain separate versioning information per-component.

Second, can you provide some links regarding Microsoft's "versioned GAC," please?

Third, would you be willing to send me the text from the relevant chapter of your book so I can better understand your last few sentences?

May 16, 2011

Yep, on board with the version the framework as a whole thing. But multiple versions of the framework should run side-by-side and be available to multiple sites on the same box In a shared library fashion

The mono guys have a good write up on GAC. http://www.mono-project.com/Assemblies_and_the_GAC

Essentially the GAC solves the DLL hell problem, in that, you can install as many versions of the DLL as you want, and the apps will reference the version they want specifically or will tag against the latest version.

I have ZFW versions on my machine running back ages...

ls -1 in /usr/share/php/ZendFramework gives

ZendFramework-0.1.5

ZendFramework-0.2.0

ZendFramework-0.6.0

ZendFramework-0.7.0

ZendFramework-0.8.0

ZendFramework-1.0.0

ZendFramework-1.0.1

ZendFramework-1.0.2

ZendFramework-1.10.3

ZendFramework-1.10.4

ZendFramework-1.10.7

ZendFramework-1.11.0

ZendFramework-1.11.2

ZendFramework-1.11.3

ZendFramework-1.11.4

ZendFramework-1.11.5

ZendFramework-1.5.0RC1

ZendFramework-1.5.1

ZendFramework-1.5.2

ZendFramework-1.5-PR

ZendFramework-1.6.0

ZendFramework-1.6.0RC2

ZendFramework-1.6.0RC3

ZendFramework-1.6.1

ZendFramework-1.7.0

ZendFramework-1.7.0PR

ZendFramework-1.7.2

ZendFramework-1.7.6

ZendFramework-1.8.0

ZendFramework-1.8.0b1

ZendFramework-1.8.0PR

ZendFramework-1.8.4PL1

ZendFramework-1.9.1

as an example.

May 17, 2011

This is already possible with the current distribution mechanism, and we've had functionality in Zend_Loader_Autoloader to accommodate this exact use case since 1.9 or 1.10.

My understanding is Pyrus also has this capability, but I'll follow up with the Pyrus developers as well to verify.

May 17, 2011

Pyrus doesn't have this ability. Like PEAR it installs everything to the same filesystem path (i.e. you can only ever have one version of a package) - and overwrites on all subsequent upgrades. It's one of the reasons I never use PEAR for live apps. Ideally, we should be able to install any version, and load any version installed, via some autoloader logic. It would remove a heck of a lot of maintenance scripting and build phases (using multiple PEAR repos locally doesn't solve any of the maintenance/scripting pain either). Also bear in mind that via-Pyrus/PEAR also raises the scenario that the ZF version in use is not a global version - there could well be a mix of different component versions. So no, Pyrus doesn't support the multi version scenario. I'm not familiar with the GAC, the best example I can think of that's easy to follow would be RubyGems. It's a bit different though - PHP might be somewhat better since we use autoloading.

Might be worth raising with the Pyrus devs...

May 17, 2011

Pretty strong vote against Pyrus then.... proper QA practices demand that upgrades not be performed in-place.

May 17, 2011

I'd be more worried about stability, I think it's premature to settle on Pyrus anyway while in its current alpha state. But, when stable, supporting local installation of multiple versions of any one package would solve a lot of problems. We shouldn't have to feel pushed into relying on git, scripting or external management tools - Pyrus does have some very good points. The fact it offers properly signed packages is itself great for those who would be thrown off a rooftop by their employers for pulling down code from insecure sources.

May 18, 2011

I'd like to note once again that we're not discussing packaging here. We already have a thread on zf-contributors regarding the location of resources, and we will discuss the distribution in yet another thread. Not on this page.

May 16, 2011

9. Security-By-Design. Need components for privacy compliance and security. Need major improvements to the Zend_Version system and to the site-administrator notification systems. These shouldn't be bolt-on, but default components. When Zend releases a User-Security-Notification, every defaults-installed ZF2 app should start whining to the system administrator that an upgrade is needed. I'd suggest a probability based system that triggers on 1% or so of hits, and can track what notifications have already been sent... set the system to send an email at least every 24hr's until the site is patched. This is critical to maintaining the frameworks reputation for security by policing the installed base. Privacy-by-design means creating a data tainting system ... and similarly SQL issues can also be controlled through data tainting. A system that addresses where data came from and where it is safe to display/use would be hugely helpful towards privacy and security. We should get rid of the current escapeFilter approach in support of something more sophisticated.

I would argue that taint mode should be a language level solution, not framework-level. Either that or something for static code analysis.

Other than that, I agree with everything else here... but can you propose some ideas for replacing the escapeFilter approach?

May 16, 2011
<ac:macro ac:name="unmigrated-wiki-markup"><ac:plain-text-body><![CDATA[There's no language level solution for data tainting that I know of in PHP.
The escapeFilter could be deprecated by a tainting model that says,
class Zend_String {
function getOrigin()Unknown macro: { //return database, form, controller etc }
and each time a string is used say in a view binding to $this->whatever instead of escapeFilter being used, it would throw an exception saying something like 'tried to bind database result directly to view. Try $field->getResultAsViewString()' or similar.
I'm not sure of the best API, but the concept of tainting data so that its origins are known, and that components wont just blindly work with tainted data is probably a good thing. Performance concerns crop up again though.]]></ac:plain-text-body></ac:macro>
May 17, 2011

Off-topic: I find it funny that you worry about hash lookups in the DI container, but then advocate a proliferation of objects like this for security purposes. <ac:emoticon ac:name="smile" />

(Not disagreeing with the approach, but had to comment.) <ac:emoticon ac:name="smile" />

May 17, 2011

Yes, I note the performance concerns... however there's not a huge distinction between instantiating a new already-loaded class and creating a new variable. With the DI thing, the concern is the repeated number of calls. But yes, performance is a critical item here.

May 16, 2011

13. Concerns regarding ORM integration. Is doctrine still going forward as the Zend_Db replacement? Are we going to better map to PDO (specifically, reworking the _quoteIdentifier mess, using proper prepared queries, array binding, etc)

No, Doctrine will not replace Zend_Db. In fact, Ralph has been working on a refactor of Zend_Db for some time now (proposals are here in the ZF2 wiki) that offers a much better roadmap for adapters, including addressing the concerns about prepared statement usage (or lack thereof in many cases). Please see his proposal, as well as the repository where he's developing the code, if you have questions or feedback about its direction.

Regarding Doctrine, we do want to provide some integration at the application configuration level to make it simple to add it to an application; however, it is by no means a replacement for Zend_Db.

May 16, 2011

19. Fix Zend_Session design issues. Make setExpirationSeconds work properly.... see point 18... integrate clean-up tasks. See ZF-4753 which is still outstanding.

Can you please review Zend\Session in the current ZF2 master branch and see if it addresses these issues? I completely rewrote it last year due to an inability to test it during the transition to namespaces.

May 16, 2011

20. Address contribution/process overhead. See http://radar.oreilly.com/2011/05/process-kills-developer-passion.html

The problem is that with a project this size, without a process we have:

inconsistent APIs

developers trampling on each other's feet as they implement different approaches to the same problem

We must have some process to manage contributions. Can it be better? Absolutely. It starts with providing concrete suggestions on making it better.

We have the CR Team now, which helps, but we can still make it better. A better workflow for submitting proposals and moving them through to acceptance into the master branch, particularly something that sends nagging emails when a proposal sits too long in a single stage, would be a huge improvement. We also need guidelines for review (which may include using CodeSniffer, phpmd, and other tools configured for our needs), and more. Want to start an RFC?

May 17, 2011

If you want to work out something, Kevin, let me know. I'll be trying to document some approach to keep proposals moving and could use the help (the CR Team is aware we need to boil things down more efficiently).

May 17, 2011

I liked the old way... solve a problem, post a patch to the mailing list... it gets vetted/changed by a buncha eyes then someone with commit rights adds it in... if you're regularly submitting patches, you get commit rights.

If someone wants to write tests and docs for a patch, great. But the status-quo of rejecting untested/undocumented code isn't a solution either.

May 17, 2011

I think the old way sort of died once we started shedding lead maintainers. Simple personal example is Zend_Feed - all my stuff, and dozens of patches, are all committed, vetted and in use for two sizeable components with very few o/s issues on the tracker. If I dropped off the planet - what would happen? Most likely, patches would linger, maintenance would slow down/stall, etc. It could be resurrected, somewhat, which is why I'm looking into how CR Team should operate and how proposals could be streamlined. Our main block is tooling and maintaining personal contact with proposers, contributors and maintainers. We'll get there - and ZF2 should pose no problems (it doesn't have a few thousand o/s issues, of course!).

May 17, 2011

...the status-quo of rejecting untested/undocumented code isn't a solution either.

The project is large and people are busy – very few people want to spend extra cycles on reviewing something that is untested even when they are getting paid to do it – why would anyone do it for free?

Sure, sometimes a very vague explanation of an issue or a patch without tests will be taken and a contributor will run with it. It takes time but we are all willing to do this sometimes; however, it is quite obvious that this approach doesn't scale and nobody can approach development this way on a regular basis.

I mean, I've contributed patches "WITH TESTS" that didn't get looked at for a while. No harm done, people are busy.

Now, can this be improved...sure; however, the project needs good suggestions on HOW to do this. I mean, if someone could wave a magic want to fix it, I'm sure they would, but it just isn't that simple.

May 17, 2011

I've given suggestions. Get rid of the process overhead. Get rid of the testing requirements. Get rid of the documentation requirements. Kill the ZFPROP process.

In essence kill the process. These tools suck. Lets go back to the way it was run pre-1.0. Someone resurrect Cal Evans's weekly summaries, lets move it all back to the mailing list, and do something radical like having Zend team members communicate through the mailing list so everyone knows what the heck is going on – you know... like we used to – and like most opensource projects do.

That will get people interested. Asking em to hit refresh repeatedly on this page wont.

May 18, 2011

Completely agree with keeping discussions on the mailing list. Linking to a wiki page for reference is fine, but discussions should not be held there.

May 18, 2011

Killing the process isn't a solution. What is the process exactly>

1. Come up with a good idea.

2. Discuss it.

3. Write a proposal.

4. Discuss it (and invite community review).

5. Request a formal review (CR Team).

6. Discuss it.

7. Wait for a final decision (Zend).

8. Discuss it.

9. Implement it (if you survived this long without doing so).

10. Discuss it.

A vast proportion of what is being called the "process" is discussing and there is no rule saying you can't do this on the mailing list. There's no rule saying you can't discuss anything else either on a mailing list. I am, in fact, strongly in favour of using the mailing list in place of a lot of what happens on IRC (which I don't view as an acceptable channel for openly discussing items of interest with the commnunity - it's just a step up from a private conversation in many cases).

Speaking of which - this topic belongs on a mailing list!

May 18, 2011

This process sucks.

My process.

1. Work

2. Improve something in ZFW as part of Work.

3. "Hey this would be cool for ZFW, want it?"

4. "Yes|NO|Yes+we're going to change the API a bit."

5. Commit.

I don't have the time or the patience for 10 levels of planning and discussion. The process you describe works for large-scale waterfall type projects, not adding patches. Its appropriate for the MVC/2.0 when everything is going to BC break discussion, but is utterly broken for the day-to-day improvements to an existing component development.

Steps 10 through 20 on the current process are write tests, docs, <ac:link><ri:page ri:content-title="play politics" /></ac:link> and then around step 19 you get to write some code.

May 19, 2011

I'm sorry, but I can't take this seriously.

If someone wants to write tests and docs for a patch, great. But the status-quo of rejecting untested/undocumented code isn't a solution either.

Without documentation, developers using the framework do not know how to use it. We already get gripes about the documentation – imagine how those will be amplified – and adoption will decrease due to the inability of developers to discover and learn functionality.

Without tests, we cannot maintain the framework long-term. The "old way" led to BC breaks - new code broke existing features or bugfixes. Testing is an absolute requirement if we want to maintain stability long-term.

I know how fun the "old way" was, but we left that way when we went stable with 1.0.0. It's simply an untenable approach for long-term maintainability or education of end-users.

May 19, 2011

If you'll recall I had pretty serious objections about the way we marched from 0.x to 1.0 too.

BC breaks happen under the current process too, but if you'll review my packaging proposals, BC breaks will be less important as developers will adopt the new versions as a course of application development rather than as the system libraries are upgraded.

In this context, new features can be added all the time, and used as they are documented. If issues come up, tests can be written.

There is more than one way to ensure reliability and code quality, doc and test driven development is not the only path.... and it has many seriously obstacles.

May 17, 2011

16. Better HTML sanitization component. Maybe look at bringing in a third-party lib that does this well?

My particular area of interest - I hope to bring forward a solution in a few months (once work eases off for the Summer). It's a particularly nasty minefield though. As you probably know from my blogging, the only working solution is HTMLPurifier so I've prototyped a library that works at the same level but improves performance significantly (the only complaint against HTMLPurifier is its speed). I can't emphasise enough how bad the alternatives are in PHP. The vast majority are simply riddled with easily discoverable vulnerabilities, a symptom of their age, lack of testing, lack of developer knowledge, and basically, lack of any decent security awareness. When I do bring forward a solution, I would expect it not to be hurried along - it's worth taking the time to have it vetted by the security community. It will take days of work for me to work through the possible hacks to ensure its sufficiently bulletproof for distribution. I think the prototype is itself elegant enough (look up Wibble on my blog) and isn't overly complex - it relies heavily on the DOM and I just about have the right balance of fixes to make DOM's quirks negligible. Before it's added, I'll remind Matthew that the likelihood of it yielding a security vulnerability down the line is 100%. Nothing weird there - all security libs eventually hit that wall where the unexpected strikes home. It will probably air outside the ZF separately simply to maximise exposure to peer review.

May 17, 2011

Let the flaming begin! (just kidding of course)

I don't like DI and wrote a long comment and then decided not to post it. In summary, this pattern to me is a registry pattern with magic under the hood.

So for me it boils down to:

How tightly coupled is DI with the rest of the framework? (Aka, do I need to use it?)

How easy is it gonna be to replace this, or to setup and inject objects myself?

How do you plan to educate developers so they know what is actually going on. "You shouldn't have used it like this!" ain't gonna cut it.

May 17, 2011

I've noticed a disconnect between the idea of DI and the idea of DI. Confused? DI (Dependency Injection) is, simply, necessary for all languages supporting OOP. You do it every day, in every class, for every library or app. So no - it's not optional. More for others, Till, I'm sure you get that <ac:emoticon ac:name="cheeky" />. The tricky part is a DI Container - a configuration vehicle. Is it tightly coupled? By definition, it can't be. Anything you could do with a DI Container you could do manually. Of course, you'll need to write all the code yourself which part of the point - a DI Container shifts the focus away from hard coded deps to configurable deps - easier for testing, easier to manipulate and less code scattered across the app (it's more centralised in a config file). In this way - it also means less maintainance and potentially fewer bugs.

Replacing it will be possible - since it's outside the class hierarchy being configured, it's decoupled. Of course, controllers and views probably will contain direct references to its interface since these will have controlling code to use the container, presumably defined by an interface you could implement.

Educating users should be kept to the absolute basics. I'd even argue that if the API feels too complex, it probably is. My own explanation would keep it simple. Objects use other objects. DI is what gets those other objects inside the using object. You can do it manually or you can do it via some configurable process, i.e. a DI Container. If the "theory" requires any other complex teaching, well, you're writing it wrong <ac:emoticon ac:name="wink" />.

May 17, 2011

Anything you could do with a DI Container you could do manually.

Can someone please setup an auto-responder so this goes out automatically to everyone on a periodic basis? Kidding of course <ac:emoticon ac:name="smile" />

May 20, 2011

Yeah, I have confused that indeed. The words, not what it is.

I guess confused that multiple times over the course of the data. That's what 140 character flame wars on Twitter do to you (or me)!

I meant to say that I'm not in favour of a DI containter. That's all. Or at least making it mandatory for components to use or within the MVC pattern.

I guess I agree on the shift you mentioned (hard-coded deps vs DI container), but this sounds like abstraction hell.

Will ZF2 power web application development, cars and nuclear power plants also? Probably not, it'll be used for web applications.

Anyway, I've added my opinion.

May 19, 2011

* How tightly coupled is DI with the rest of the framework? (Aka, do I need to use it?)

DI is a layer on top of the application, used to wire up dependencies. In the prototypes, I use a DependencyInjector instance to build an explicit ServiceLocator class – which is something I could write by hand if desired.

The main use case is to grab a controller and its dependencies within the front controller. Because we will be using simple, clear interfaces, you should be able to write an alternate front controller easily and quickly should you want to manually wire dependencies at the top level – and nothing will prevent you from simply doing as you've always done in ZF1.

You can also easily replace what implementations you want to use within your configuration – which is not unlike how we do things in ZF1 with Zend_Application.

* How do you plan to educate developers so they know what is actually going on.

The same way we have with Zend_Application. You show them first simply using the features, and then go into the details of how it works once they're already using it. Look at the symfony2 DIC documentation for a great example showing how easy it is to actually describe the concepts. It's not rocket science, and the concepts are actually fundamentally easy to understand. We just need to make sure we get good feedback on early documentation to ensure it speaks to a variety of audience levels.

May 20, 2011

Yeah, I looked at Symfony a lot and I've decided to stick with Zend Framework. <ac:emoticon ac:name="wink" />

Anyway, I'm curious to see how things turn out without it.

Just let me say this: I really don't mean to discredit your or anyone else' work, but I just don't agree on the success of Zend_Application.

It's part of the reason why people are confused and why a standard setup is considered slow by many people.

I guess this is one of those 90/10 things where it works for most people, but once you're (stuck) with it, taking it out is an insane task.

Also debugging (and also profiling) an application is overly complex task, once you have to step through all the magic Zend_Application does to abstract and hide boilerplate from the general developer.

May 18, 2011

I'm beginning to wonder if some form of templating system should be in place by default. One aspect that I don't quite like in ZF1 is that I/we ended up initializing default View properties in the bootstrap, e.g default scripts, css and meta info. Which is unnecessary for say ajax requests. But then the question is where would be the best place to do this such that (ZF1's) view templates can add or amend these properties (e.g meta tags). Which to me suggests that if the layout.phtml was governed by a templating engine then the defaults could be cleanly/easily specified there?

May 18, 2011

Having hacked Zend_Controller for two days, I found something else we should have: The ability to change dispatcher /after/ having determined the active route.

May 18, 2011

Can you be more specific... what's the 80% use case for this?

May 18, 2011

I'm fairly sure neither 80% nor 20% would use this ability. I'm not saying ZF should supply multiple dispatchers, nor am I saying it should have some built in way of changing it, it should simply have the ability to implement this functionality. Currently in the Zend_Controller_Front::dispatch(), the dispatcher is retrieved first, then the route is determined. It's pretty much impossible to change that in current states. All I'm saying is that there should be an ability to set the dispatcher to use /after/ the route has been determined.

In this specific case we use a separate dispatcher for REST routes, but have to determine first whether it's a rest route that's active - or not. I suppose the rest part will be better handled in ZF2, but I suppose someone else will then think of a different use case in which this still may prove useful.

==

If we get the rest solution generic enough I may blog about it, and see if we can get it FOSS.

May 19, 2011

The question is whether or not we'll have a separate dispatcher. Since a "controller" is anything dispatchable, we could simply have different base action controllers that implement different types of logic. The zf2sandbox prototype shows this with both RESTful and regular action-based controllers.

May 20, 2011

Of course, the above is assuming there will be a separate dispatcher. If there wont be, we dont need to be able to switch it <ac:emoticon ac:name="wink" /> (I think there's gotta be some logic reasoning behind it).

On a different note, it goes without saying (I think) that a method like zend_controller_front::dispatch() should not be a 150 lines long in a zf2...

May 20, 2011

Take a look at this, then: http://git.mwop.net/?a=viewblob&p=zf2sandbox&h=ead2f05f042dfe646f439b68b92debde3dcd9648&hb=5a5ca1b518e36f5fa40366d3814321082222bb49&f=library/mwop/Controller/Front.php

Interestingly, there's code in there that can be stripped at this point (the "controller map" bit is redundant when we're pulling controllers from the DI container).

Also, browse that tree and look for the restful controller implementation, as well as the "page" controller – they better show off the ideas of moving the mapping of actions into the controllers themselves.

May 28, 2011

A variable configuration system for the routing will be nice where we can put specific config variables for the routes, and router will use these variables to create/map the routes.

For example, I have a module named "store" and I created some routes for the module, but then I have to use "storefront" as module name instead of "store", so I need to change all the routes I have defined with "store/?" into "storefront/" manually. So what if we have the option to add variables in the route and a system where we can assign values to these variables. and I will define routes like "<module_variable_name>/???" and in the config I will just provide the value for <module_variable_name> which can be "Store" or "Storefront" or "Shop" etc.

Jun 13, 2011

Zend_Application

I have to say I enjoy Zend_Application and really want to see it in ZF2. My complaints about Zend_Application are it is SLOW! I believe in our current application it can take up to 200ms just for the bootstrapping process. Part of this is due to the standard implementation runs all of the elements in the bootstrap.

A way to pull this out and gain the flexibility and speed is to move some of it to a dispatching hook. An example can be utilizing the class to specify what elements need to be bootstrapped:

<ac:macro ac:name="code"><ac:plain-text-body><![CDATA[// snip from the bootstrap harness (aka index.php)
$app = new \Zend\Application(APP_ENV, APP_PATH);
// snip from a controller public property
$bootstrap = array(
'myAction' => array('database', '...');
// snip from a front controller plugin
foreach ($controller->bootstrap['myAction'] as $boot) {

Another thing that I think is missing is global access to the bootstrap in a controlled way which allows to retrieve the elements quickly and easily.