Recently I created a model I called sfDoctrineSecurityGenerator. It’s an awesome class that you can drop directly into generator.yml that matches the admin generator’s security credentials with those that you set in security.yml. This works with the default admin generator in core, and has worked swimmingly for me.

So what is sfModelGenerator? sfModelGenerator is an abstract class extended by the ORM in order to auto-generate files. This class is used anywhere you have a generator.yml file, or any time you run the tasks ./symfony doctrine:generate-module or ./symfony doctrine:generate-admin.

Step 0 (optional): If you don’t have a module set up that uses the generator.yml file, see Fabien’s Screencast on how to use it. You can also run a command like this: php symfony doctrine:generate-admin myapp MyModel.

Step 1: Add this file to /path/to/project/lib/generator/sfDoctrineSecurityGenerator.yml.

Step 2: Swap out the sfDoctrineGenerator field in your generator.yml for sfDoctrineSecurityGenerator. Should look like this:

generator:
  class:                            sfDoctrineSecurityGenerator      # was "sfDoctrineGenerator"
  param:
    model_class:                     MyModel
    theme:                              default
    ...

Step 3: Add a security.yml file to your module’s config directory if one doesn’t exist already. This would look something like below:

# /path/to/project/myapp/modules/company/config/security.yml
all:
  is_secure: on

edit:
  credentials: [Company Edit]

new:
  credentials: [Company New]

delete:
  credentials: [Company Delete]

Step 4: Set the use_security_yaml_credentials flag to true.

Add use_security_yaml_credentials: true to your generator.yml’s params, so it looks like the following:

generator:
  class: sfDoctrineSecurityGenerator
  param:
    model_class:           Company
    ...
    use_security_yaml_credentials: true

    config:
      actions: ~
      fields:  ~
      list:    ~
      filter:  ~
      form:    ~
      edit:    ~
      new:     ~
Note: If you don’t set up a security.yml file (Step 3), this next step will throw an exception

You are now in business. The admin generator is now smart enough to hide EDIT links from users lacking the Company Edit credential.

This is the same with DELETE and NEW actions as well. In fact, this works for any custom action declared in generator.yml. For instance, if your generator.yml looked something like…

...
  config:
    actions:
      mycustomaction: { name: 'Do Stuff Now!', action: dostuff }
...

…and a dostuff credential existed in security.yml, the link “Do Stuff Now!” will only show to those with the correct credentials. Pretty cool stuff.

The sfDoctrineSecurityGenerator.yml class can be found here. As always, please report any issues and give me your feedback!

A very useful feature is the ability to pinpoint which queries are being called where. Especially if you have one query taking 6 seconds to run. Symfony and Xdebug will provide a nice little arrow next to the query in your debug toolbar, providing a stack trace from when it was run.


Add this code at the bottom of your application’s factory.yml:

# myapp/config/factories.yml
dev:
 logger:
   param:
     loggers:
       sf_web_debug:
         param:
           xdebug_logging: true

The disclaimer is this does account for a hit on your performance, hence why it was disabled by default in releases 1.3.2 and 1.4.2. I would recommend keeping it disabled until it comes time to optimize your queries.

Here’s an easy way to add ordering by an array in your doctrine query. This is only useful in a handful of use cases, namely when a query is receiving a list of IDs in the order they should appear. The only way to order them correctly would be via a method like this.

An example of this would be if a request was made for the ids 12, 224, 1, and 14. How are you going to order your result set correctly? Like this:

  $ids = array(12, 224, 1, 4)

  $query->whereIn('id', $ids)->orderBy('id', $ids)->execute();

That’s it!

The actual SQL is a bit more complicated, and uses the MySQL FIELD function like so:

  SELECT * from my_table where id in (12, 224, 1, 4) ORDER BY FIELD(id, 12, 224, 1, 4) = 0 ASC, FIELD(id, 12, 224, 1, 4) ASC

Yes, this does work when using the addOrderBy function. Yes, you can pass ASC and DESC values to ORDER BY. And yes, you can use other column types in addition to primary keys / integers.

If you’re wondering why the query above contains TWO FIELD calls, here’s why: The first FIELD call is there so that any result not matched in the ORDER BY array is sorted after those matched in the array. I made this adjustment because it seems intuitive to me that if you don’t supply a whereIn argument, the passed ORDER BY array of values is first, followed by however else ordering is deemed necessary.

You can view the code on github. Thoughts?

Disclaimer: This has only been tested using a MySQL database

Who hates migrations? If you have used migrations to any extent, your hand better be up. A lot of migration-greif comes from this fundamental flaw: Migrations are static, your application is dynamic. Yet migrations depend on the application in order to run. The following issues are examples of this phenomenon:

  1. Failing migrations as a result of a removed column
  2. Failing migrations as a result of methods being called that no longer exist
  3. Extensive use of SQL in your migrations to avoid loading Doctrine objects
  4. Loss of bowel control at the mentioning of “Doctrine Migrations”

These issues can consume development hours, leave your development team in disarray, and cost a fortune in new boxer briefs. The simple solution: Sandboxing

At the Ruby on Rails Meetup last thursday, I saw a revolutionary piece of code that blew my PHP-paradigmed mind:

class MigrateFoo < ActiveRecord::Migration
  # this is significant!
  class Foo < ActiveRecord
    # declare what the migration needs
  end

  def self.up
    # do up logic
  end
end

Let me unpack this. The model being migrated (Foo) is namespaced inside the migration. When Foo is referenced inside the migration, the namespaced model MigrateFoo::Foo is instanciated. You can now declare everything you need in the migration, to separate your migration's dependency on your application, and it will always be there when the migration needs it. We can use this concept in PHP / Symfony / Doctrine as well. Here is an example situation:

  • We have a model called Book
  • We have a model called Author
  • A One-To-Many relationship exists between Book and Author
  • We realize after launching that we need a Many-To-Many relationship between the two, in the case of co-authorship (duh)

Now, in order to migrate, we need to

  • first: add the new link table
  • second: move all existing author_ids out of the book table and into the book_author table
  • third remove the now-deprecated author_id field from the book table

This is all fine until you run into one small (enormous) problem with step two. If you migrate before you build your models, doctrine is not aware of the book_author > author_id column, and will throw an error. If you migrate after you build your models, doctrine is not aware of the book > author_id column, and will throw an error. Its an old-fashion Damned-If-You-Do-Damned-If-You-Don't scenario. At no point in our application timeline will our models have both relationships. However, our model needs to be aware of both these relationships in the migration in order to work with the data.

In the past, I've executed straight SQL in my migration in order to bypass the ORM entirely. This works fine in some situations, but what if you have complex queries already built to handle the implementation logic? What if you need to use the ORM?

Solution: Sandbox those migrations!

This is actually a very simple problem to fix. All you need to do is sandbox your migrations. After running php symfony doctrine:generate-migration migrate_book_author_data task, I should have a file named something like /path/to/project/lib/migration/doctrine/1274632922_migrate_book_author_data.php. Perfect! Now, lets add the following models:

class Migration_1274632922_Book extends Book
{
  public function setUp()
  {
    parent::setUp();
    $this->hasColumn('author_id', 'integer', '4', array('length' => '4', 'type' => 'integer'));
  }
}

class Migration_1274632922_BookTable extends BookTable
{}

If you're running PHP 5.3, I would definitely recommend namespacing the classes rather than adding prefixes. For example, \1274632922\BookTable, or even \Migration\1274632922\BookTable, to keep your code cleaner.

Now, we cannot add these classes directly in our 1274632922_migrate_book_author_data.php file. Doctrine interprets every class as a migration class, regardless of whether or not it extends Doctrine_Migration_Base. I put them in /path/to/project/lib/migration/model/1274632922/Book.php. Now, include your extended class files in the methods of your migration, and you can migrate as expected.

Note: At the time this migration is run, we assume a previous migration has already added the book_author link table. We have both the book_author table and the author_id field simultaneously present in our database schema.

class migrateBookAuthorData extends Doctrine_Migration_Base
{
  public function preUp()
  {
    include_once(dirname(__FILE__).'/../model/1274632922/Book.php');

    // Fix Data
    $books =  Doctrine::getTable('Migration_1274632922_Book')->findAll();

    foreach ($books as $book) {
      if ($book['author_id']) {
        $bookAuthor = new BookAuthor();
        $bookAuthor['author_id'] = $book['author_id'];
        $bookAuthor['book_id'] = $book['id'];
        $bookAuthor->save();
      }
    }
  }

  public function up()
  {
    $this->removeColumn('book', 'author_id');
  }

  public function down()
  {
    $this->addColumn('book', 'author_id', 'integer', '4', array());
  }

  public function postDown()
  {
    include_once(dirname(__FILE__).'/../model/1274632922/Book.php');

    // Fix Data
    $book =  Doctrine::getTable('Migration_1274632922_Book')->findAll();

    foreach ($books as $book) {
      if ($book['Authors']->count()) {
        $author = $book['Authors']->getFirst();
        $book['author_id'] = $author['id'];
        $book->save();
      }
    }
  }
}

Our migration moves the ids from the book > author_id column to the book_author > author_id column in the preUp() method, and then removes the book > author_id column in the up() method. We do the exact opposite in the down method.

By extending the object and calling the extended object in our migrations, we no longer care what fields are in our application-level models. Our migrations are sandboxed, and will always have the methods/columns it needs to carry out the migration successfully.

How many of you encounter the need for complex where statements in Doctrine? What application DOESN’T require a complex where statement on occasion? Well, what I’m getting at is most of them do!.

If and when you encounter such a situation, you can choose one of three paths (but choose wisely):

  1. - Break your query into smaller, more manageable queries and execute them separately
  2. - Drop down to DQL, or even straight SQL (essentially bypassing the ORM layer) for more granular control and to utilize options not available in the Doctrine_Query object (subqueries, etc).
  3. - EXTEND THE DOCTRINE QUERY OBJECT!

One and Two are great options, and should always be considered. However, there are occasions where a query object is just more convenient. For example, if you are using the default symfony admin generator, you should be aware its core functionality requires a Query object. And hey, what’s the use of writing all this object oriented code if we can’t subclass every now and then? Plus, it’s a lot of fun.

Predicament

I want to filter a query (any query) by a date range. I want to take whatever timestamp fields are tied to this object, and make sure at least ONE of them falls in that date range. I call the filter “Occurs In Range”. Here’s an example:

  • - I have a list of assigned tasks
  • - Each Task object has an assignment date, a due date, and an action date.
  • - I want to filter my tasks by a date range, and find any tasks where any of those dates fall between my range

Task 1:

  • title: Launch that website for that friend
  • description: Jack wants a website professing his love for his cat. He wants you to make it for him.
  • start date: April 1st, 2010
  • end date: April 30th, 2010
  • action date: null

Task 2:

  • title: Purchase Food for St. Patrick’s Day Party
  • description: You are having a wild party for St. Patrick’s Day. Don’t forget cabbage!
  • start date: March 1st, 2010
  • end date: March 16th, 2010
  • action date: March 17th

How are we going to write a filter that adds this query? Sure, we could iterate over the timestamp fields, adding ->orWhere’s. But if we have any where clause already attached to this query, it’s going to screw with the logic. Why would you let Doctrine screw with the logic? I tell you the truth brother, this is a thing of evil.

Solution

By subclassing the Doctrine_Query object, we can create our own query. I called mine Doctrine_Query_Extra, although other good suggestions were Doctrine_Pimp_Query, Doctrine_Bling_Query, and Bling_Pimp_Doctrine. I then added a few methods to allow us better handling of our where clauses. These functions include andClause for beginning a parenthetical where clause with AND, orClause for beginning a parenthetical clause with OR, and endClause for ending either clause. Here is an example of these functions in action.


  public function addOccursInRangeColumnQuery(Doctrine_Query $query, $field, $value)
  {
    $query->andClause();

    foreach ($this->getOption('occurs_in_range.filter_fields') as $i => $f)
    {
      $this->orClause()
          ->addDateQuery($query, $f, $value)
          ->endClause();
    }

    $query->endClause();

    return $query;
  }
 

How cool is that? We in effect just wrapped our entire filter in an AND statement, and made sure our individual date query statements were OR’d (i.e. only ONE of the object’s dates need to be in the range). Shazam.

Some of you are confused with the above function. “What is addOccursInRangeColumnQuery”? Quite simply, if you set an ‘occurs_in_range’ filter field to the “sfWidgetFormFilterDate” widget in your Form Filter class (something like *Model*FormFilter.class.php) and also a “occurs_in_range.filter_fields” option to your array of date fields:

  $this->setOption('occurs_in_range.filter_fields', array('start_date', 'end_date', 'active_date'));

…Doctrine and symfony will automatically look to call that method when that filter is used. Put the above code in your BaseFormFilterDoctrine class and you’ll be ship shape. Take a peak at my Code Snippets link at the bottom to see this in context.

In Doctrine 1.2+ you can tell Doctrine to use your subclass by calling

$manager->setAttribute(Doctrine_Core::ATTR_QUERY_CLASS, 'Doctrine_Query_Extra');

in your project configuration. If you are using less than Doctrine 1.2, you can create the Doctrine_Query_Extra class by calling Doctrine_Query_Extra::create(), or by overloading the Doctrine::getTable('TableName')->createQuery() method on a table by table basis, which will look something like this:

    public function createQuery($alias = '')
    {
        if ( ! empty($alias)) {
            $alias = ' ' . trim($alias);
        }
        return Doctrine_Query_Extra::create($this->_conn)->from($this->getComponentName() . $alias);
    }

You can download this class using the link below. If you use it, let me know if it works well for you, and please post here if you can think of ways to improve it!

Links

**Update**
Fixed an issue in Doctrine_Query_Extra where an error was thrown for empty clauses (i.e. $query->andClause()->endClause()). This is now fixed.