Tuesday, October 22, 2013

TYPO3 - PHPUnit Testing Framework tests for repositories with storagePids

When I develop a TYPO3 ExtBase extension, I often use the Testing Framework shipped with the TYPO3 extension PHPUnit. With the Testing Framework it is possible to create real testrecords in the TYPO3 database and use those records in your tests to verify everything works as expected.

In a project I faced the situation, that one of my repository methods did not find the records created with the Testing Framework. Well, after some hours of debugging I finally found the solution.

The problem

I created a new TYPO3 ExtBase extension with some tables, a controller and a view. Really nothing special about that. I then created a function in my repository which searches for some records in the database with a configurable limit. 


/**
 * Returns the latest records. 
 *
 * @param int $limit The Limit
 * @return array|\TYPO3\CMS\Extbase\Persistence\QueryResultInterface
 */
public function findLatest($limit) {
 $query = $this->createQuery();
 $query->setOrderings(array('datecreated' => QueryInterface::ORDER_DESCENDING));
 $query->setLimit((integer)$limit);
 $records = $query->execute();
 return $records;
}

Then I started to create some tests for the newly created function in the repository. One test was as following.


/**
 * Test if findLatest returns expected amout of records if limit given
 *
 * @test
 * @return void
 */
public function findLatestReturnsLimitedResult() {
 $this->testingFramework->createRecord('tx_myext_domain_model_table', array('datecreated' => time()));
 $this->testingFramework->createRecord('tx_myext_domain_model_table', array('datecreated' => time()));
 $this->testingFramework->createRecord('tx_myext_domain_model_table', array('datecreated' => time()));

 $this->createTestRecords();
 $this->assertEquals(2, $this->myRepository->findLatest(2)->count());
}

In the test above, 3 records with the Testing Framework are created and I'll use the function findLatest with a given limit of 2 to test, if really 2 records are returned.

Well, actually findLatest did return 0 records. I then created some real records in the TYPO3 backend and verified, that findLatest actually works as expected, so the problem must be somewhere in my test or in the TYPO3 setup.

So I started to debug my test and first of all, I verified, that the Testing Framework really created 3 records. Yes, 3 records were created, all having the pid 0. Ok then, since the testrecords where created correctly, there seem to be something wrong with the search. I'll then debugged the resulting SQL Query from the repository function findLatest and found out, that the records were searched in pid 1. OK, so the problem seems to have something to do with the storagePids.

ExtBase, tests and storagePid(s)

When you create an ExtBase extension, the Extension Builder automatically creates some TS for you where you can set the storagePid(s) for your extension.


persistence {
  # cat=plugin.tx_myext//a; type=string; label=Default storage PID
  storagePid =
}

This storagePid can be used to set the location of your records in the TYPO3 backend. Generally every TYPO3 Extension you create with the Extension Builder makes use of the storagePid(s) and overrides them, if a startingPoint is set in the plugin.

When you run your tests in TYPO3 backend (or through your IDE), ExtBase uses the backendConfigurationManager to get the ExtBase framework configuration. In the backendConfigurationManager class, there is a function called getDefaultStoragePid(), which returns the local function getCurrentPageId(). Let's have a look at this function.

protected function getCurrentPageId() {
 $pageId = (integer) \TYPO3\CMS\Core\Utility\GeneralUtility::_GP('id');
 if ($pageId > 0) {
  return $pageId;
 }
 // get current site root
 $rootPages = $GLOBALS['TYPO3_DB']->exec_SELECTgetRows('uid', 'pages', 'deleted=0 AND hidden=0 AND is_siteroot=1', '', '', '1');
 if (count($rootPages) > 0) {
  return $rootPages[0]['uid'];
 }
 // get root template
 $rootTemplates = $GLOBALS['TYPO3_DB']->exec_SELECTgetRows('pid', 'sys_template', 'deleted=0 AND hidden=0 AND root=1', '', '', '1');
 if (count($rootTemplates) > 0) {
  return $rootTemplates[0]['pid'];
 }
 // fallback
 return self::DEFAULT_BACKEND_STORAGE_PID;
}

So, if your TYPO3 installation where you develop your ExtBase extension has a root page, then this page ID is returned. Or if you have no root page but a TS Template which is marked as root template, then the page ID of that page is returned.

The returned page ID is used in the queryFactory class to set the storagePid(s), which leads to the situation I described earlier.

Solution / Workarounds

So how do we tell TYPO3 to search for our test records in pid 0 during tests? On solution is to set the storagePid for the ExtBase Framework to 0 like shown below.

config.tx_extbase {
  persistence {
    storagePid = 0
  }
}

This solution has some problems, since it can be overridden by other extensions and also it requires manual configuration when setting up your testing environment.

You could also just disable the check for storagePid(s) (with setRespectStoragePage(FALSE)) in your tests, but I think this solution is'nt really good, since your tests then also may find records you created in the TYPO3 backend during development.

If you want to keep your tests independent, I would recommend to set the storagePid(s) directly in the setup of the tests.

/**
 * Set up for test
 *
 * @return void
 */
public function setUp() {
 $this->testingFramework = new \Tx_Phpunit_Framework('tx_myext');
 $this->fixture = $this->objectManager->get('Derhansen\\MyExt\\Domain\\Repository\\MyRepository');

 /** @var $querySettings Typo3QuerySettings */
 $querySettings = $this->objectManager->get('TYPO3\\CMS\\Extbase\\Persistence\\Generic\\Typo3QuerySettings');
 $querySettings->setStoragePageIds(array(0));
 $this->fixture->setDefaultQuerySettings($querySettings);
}

The code above sets the storage pid to 0 and now the tests should find the records created with the testing framework and tests should not interfere with other records created manually.

Sunday, September 15, 2013

Blocking brute force attacks against the TYPO3 backend login using ModSecurity

Since the last few weeks I have noticed a constantly growing amount of failed TYPO3 backend login attempts on websites I administer. It seems I'm not alone with this problem, since a few other TYPO3 users are reporting the same issues. The method always seems to be the same - someone is doing a brute force attack against the password of the TYPO3 backend user "admin". The attack seems to be automated, takes several hours and comes from various IP addresses and from different countries in the world.

If you do not have any TYPO3 backend account with the username "admin", you could just ignore the case that someone is trying to hack your TYPO3 installation. Well, since I have configured a [BE][warning_email_addr] my mailbox gets flooded with thousands of emails about failed TYPO3 login attempts.

But what is the best way to handle and block those attacks? When I faced the first attack, I just denied the IP address of the attacker into the local .htaccess file of the TYPO3 installation. This method is implemented very quick, but always requires manual steps and is only valid for a given IP address. In the austrian TYPO3 forum I have read about a TYPO3 extension, which extends the TYPO3 backend login and is able to automatically blacklist IP addresses for failed login attempts. This idea seems to be good in general, but when you administer just more than one TYPO3 installation, the installation and configuration process of the extension can be very time consuming. I finally ended up with the solution, that once again ModSecurity seems to be the best way to handle this kind of attacks to TYPO3. During research I found an article about brute force protection using ModSecurity. Sadly the rules did'nt fit out of the box into my setup, so I had to create my own ruleset.

Below follows a ModSecurity ruleset, which blocks the IP address of an attacker for 10 minutes, if there were more than 5 failed TYPO3 authentication attempts from that IP address. Please note, that the ruleset only was tested with ModSecurity 2.7.x together with the OWASP ModSecurity Core Rule Set.

<IfModule mod_security2.c>

    #
    # TYPO3 backend login brute force protection
    #
    <Location "/typo3/index.php">
        # Turn on security engine (if disabled)
        SecRuleEngine On

        # Deny IP address if too many TYPO3 authentication failures
        SecRule IP:bf_block "@eq 1" \
        "id:'5000102', \
        msg:'IP address temporarily blocked due to too many TYPO3 authentication failures', \
        phase:2, \
        log, \
        deny"

        # Check for TYPO3 authentication failure and increment counter
        SecRule RESPONSE_BODY "LOGIN_ERROR### begin" \
        "id:'5000100', \
        msg:'Failed TYPO3 authentication attempt', \
        phase:5, \
        t:none, \
        setvar:IP.bf_counter=+1, \
        expirevar:IP.bf_counter=3600, \
        nolog, \
        pass"

        # Check for too many failures from a single IP address
        SecRule IP:bf_counter "@gt 5" \
        "id:'5000101', \
        msg:'Too many TYPO3 authentication failures from IP address', \
        phase:5, \
        t:none, \
        setvar:IP.bf_block, \
        setvar:!IP.bf_counter, \
        expirevar:IP.bf_block=600, \
        log, \
        pass"

    </Location>

</ifModule>

The ruleset creates a counter (IP.bf_counter) for each IP address, when a failed TYPO3 login is monitored. A failed login is identified by the string "LOGIN_ERROR### begin" in the response body. I don't check for the occourence of the string "Your login attempt did not succeed", since some TYPO3 installations may have a e.g. german or danish backend login and within this the resulting error message will be translated and the rule would not match.

I've decided not to log each failed login attempt, since it fills up the ModSecurity audit log with the complete response from the TYPO3 backend login. If the counter is greater than 5, the counter is unset and a new variable (IP.bf_block) with a lifetime of 600 seconds is defined. Depending on this counter, the IP address gets blocked with a 403 error.

If you have blocked your own IP address during to too many failed login attempts, you have to unset the counter variable by modifying your ruleset. This is not very practical, so another possibility to unset the variable is to simply delete the file /tmp/ip.pag - this is where ModSecurity stores all variables in a default setup. Note, that deleting the file will remove all variables set by ModSecurity. Also I don't know, if deleting the file will result in any other unwanted side-effects, so be carefull with that.

Besides using ModSecurity to protect TYPO3 websites against known and unknown attacks, I also recommend using a Yubikey to add two-factor authentication to TYPO3 admin accounts.


Wednesday, August 7, 2013

TYPO3 Error "Sorry, but an error occurred while connecting to the server." in Extension Manager

After the latest security updates for some TYPO3 extensions I was about to update those extension in a TYPO3 installation. After opening the extension manager, TYPO3 (version 4.7) did show the following error message:

Connection Problem
Sorry, but an error occurred while connecting to the server. Please check your network connection.

First I thought, there was a problem with some TYPO3 cache files, but after cleaning the typo3temp folder, the problem still occured. I then looked into to Webservers Error log and found the following logentry.

PHP Fatal error:  Cannot unset string offsets in /path/to/typo3/sysext/em/classes/tools/class.tx_em_tools.php on line 382

After some research I finally found the solution to the problem, which is the result of a bug in the TYPO3 Extension Manager, which has been fixed for some days ago.

I manually opened the file "ext_emconf.php " of every installed TYPO3 extension and checked, that the constraints-array was correct. In my case it was powermail which had a wrong entry in "ext_emconf.php".

I had to change:
'conflicts' => '',
to this:
'conflicts' => array(),

After I fixed the wrong entry, the extension manager worked fine again.

Sunday, July 21, 2013

TYPO3 ExtBase backend module with progressbar using AJAX


Assume you have created a TYPO3 ExtBase Extension and must enable the user to import data (e.g. CSV file) to the tables of your TYPO3 ExtBase extension. With ExtBase, you can easily create a TYPO3 backend module so the user can upload the CSV file and start the data import.

Now assume the user uploads a very big set of data (several 100 MBs) to be imported and starts the data import process. Well, the user will see the hourglass a long, long time and if the user is impacient, he will surely assume that something went wrong and will click somewhere else in the TYPO3 backend and within that cancel the data import.

So would'nt it be nice if you could show the user a progressbar or an updating informal text, that the import is still running?

When you look at the concepts of some extensions with backend modules in TER, you will often see, that PHP methods like flush() or ob_flush() is used to update data in a backend module for long taking processes.

With ExtBase you can't use flush(), since the output for the backend module is rendered with fluid and the output will first display, when the action that displays the content has finished.

In this article I will show how to use AJAX in a TYPO3 ExtBase backend module to show an updating JQuery UI progressbar.



Please note, that the example code located on Github has been created with TYPO3 6.1 and uses namespaces, so it only runs with TYPO3 > 6.x. But - the shown technique can also be adapted to TYPO3 4.5.

First of all, I've created a TYPO3 extension with the extension builder. The extension has a backend module called "mod1". This module uses the controller "Example" and the action "index".

/**
 * Registers a Backend Module
 */
\TYPO3\CMS\Extbase\Utility\ExtensionUtility::registerModule(
 'derhansen.' . $_EXTKEY,
 'web', // Make module a submodule of 'web'
 'mod1', // Submodule key
 '', // Position
 array(
  'Example' => 'index',
  
 ),
 array(
  'access' => 'user,group',
  'icon'   => 'EXT:' . $_EXTKEY . '/ext_icon.gif',
  'labels' => 'LLL:EXT:' . $_EXTKEY . '/Resources/Private/Language/locallang_mod1.xlf',
 )
);

The extension builder automatically creates Typoscript files with constants and setup settings for the backend module templates. Those settings look like:


templateRootPath = EXT:extbase_bemodule_ajax/Resources/Private/Backend/Templates/

As you may notice, the templates, layouts and partials are located in a subfolder named "Backend". Using this, you always need to include the extension's Typoscript in your sites Typoscript template. If you remove the folder "Backend" and just place your templates, layouts and partials in the "normal" folder-structure of an ExtBase extension, there is no need to include the Typoscript of your extension. I have done this to keep things simple.

Next I've created the layout and the template for the backend module. Notice, that I have created a viewhelper named IncludeJQueryViewHelper, which automatically includes JQuery and JQueryUi with the nesceccary CSS files to the backend module.


{namespace h=derhansen\ExtbaseBemoduleAjax\ViewHelpers\Be}

<h:IncludeJQuery jquery="1" jqueryui="1" />

<f:be.container loadExtJs="0" loadPrototype="0" loadScriptaculous="0">
...
</f:be.container>

Also notice, that the f:be.container viewhelper is configured to disable extJs, prototype and scriptaculous.

In the template for the backend module I have placed some text, a JQuery UI Progressbar and a JQuery UI Button. I also created some Javascript code, which starts starts the progressbar and also retrieves the status of the progressbar by AJAX.

When the start-button is clicked, an AJAX request (startLongProcess) is called which starts a long taking process in our ExampleController. The AJAX request is asynchronous, so the process is not blocking the TYPO3 backend. Right after the long taking process is started, another AJAX request (checkLongProcess) is called which retrieves the status of the long taking process and updates the progressbar.

To keep things simple, the long taking action in the ExampleController just executes a for-loop 20 times and does a sleep(1) on each iteration. Below follows both the startLongProcess action, which starts the long process and the checkLongProcess action, which is used to update the progressbar.


/**
 * Example for a long process (just loops 20 seconds). Returns TRUE if process is finished
 *
 * @return bool TRUE if process is finished
 */
public function startLongProcessAction() {
 for ( $i = 1; $i <= 20; $i++) {
  /* Increase counter stored in session variable */
  session_start();
  $_SESSION['progval'] = $i * 5;
  session_write_close ();
  sleep(1);
 }

 /* Reset the counter in session variable to 0 */
 session_start();
 $_SESSION['progval'] = 0;
 session_write_close ();
 return json_encode(TRUE);
}

/**
 * Checks the status of the long process
 *
 * @return int Status of process in percent
 */
public function checkLongProcessAction() {
 session_start();
 if (!isset($_SESSION['progval'])) {
  $_SESSION['progval'] = 0;
 }
 session_write_close ();
 return json_encode($_SESSION['progval']);
}

Notice, that this example uses PHP session variables to store the actual state of the long taking process. You can also use a database or other techniques to store the actual state.

When working with PHP session variables inside a method, you have to keep in mind, that the session variable is stored, after the method is finished to prevent concurrent writes to the session. To store the session variable while the for-loop is running, you have to use session_write_close() to actually store the session variable.

Finally I updated ext_tables.php so the 2 new actions can be called by the module.

Conclusion
Creating a ExtBase backend module that uses AJAX to dynamically update content on the modules page is quite simple. You can use the same techniques for backend modules as for TYPO3 frontend plugins.

The complete code for the example shown here is available on Github

Monday, July 15, 2013

Direct Mail with images as table-output in TYPO3 6.x

With TYPO3 6.x, "renderMethod = table" seem to be deprecated and does'nt seem to be fully supported anymore. For sites using Direct Mail, this could be an issue, since one of the best ways to keep HTML newsletters compatible with most of all e-mail-clients is to render the output with tables. Especially when it comes to images in newsletters, it is best to use a table layout to be sure, that the images are positioned as expected.

I had to setup a TYPO3 6.x site with Direct Mail and faced the issue described above. Also images where not displayed in the newsletter, which also seems to be an issue when you use Direct Mail in combination with TYPO3 6.x

To handle those issued the following steps are necessary:

  1. For your newsletter TS template, do not include "CSS Styled Content", but "CSS Styled Content TYPO3 v4.7". This version of CSS Styled Content still includes the table rendering method.
  2. Include config.absRefPrefix = {$plugin.tx_directmail_pi1.siteUrl} in your newsletter TS Setup. This fixes the problem with images not being displayed.
After using the settings decribed above, I could successfully send HTML e-mails using Direct Mail in TYPO3 6.x. The solution has been found using TYPO3 forge, where both issues already have been reported.


Sunday, June 9, 2013

Using TYPO3 content layouts to create Jquery image sliders or galleries

For some time ago I've write an article about how to create a JQuery Cycle image slider in TYPO3 by using the media field in TYPO3. The shown technique works fine for header sliders, but what if you need to enable the website editor to dynamically add an image gallery or slider to the website? You can  try to find a good image gallery for TYPO3 in TER or you could try to integrate the JQuery Image gallery od slider of your choise without using an extension.

In this article I will show how integrate a JQuery image slider into TYPO3 by using the layout field of TYPO3 content elements without the need to install an extension. The technique shown can be used to create custom HTML output for a lot of JQuery image galleries or sliders available. In this example I will show how to create the needed HTML output for the bxslider.

Prerequisites
You need a working TYPO3 6.1 (lower versions should work as well) installation of TYPO3 with a working template and css_styled_content installed, so TYPO3 content elements are shown on the websites output.

Integration into TYPO3
First, you need to include a version of JQuery and the image slider (both JS and CSS files) to the output of your website. Upload the JS and CSS files for the image slider to a folder in your fileadmin and add the following to your TS template

page.includeJSlibs {
  jquery = http://ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js
  jquery.external = 1
  bxslider = fileadmin/templates/js/jquery.bxslider.min.js
}

page.includeCSS {
  bxslider = fileadmin/templates/res/jquery.bxslider.css
}

After adding JQuery and the slider to your frontend output, you need to add the JS for the slider to create an instance of it for each object that should be treated as an image slider. For the bxslider, you just need some lines of code. I create the file slider.js in the directory fileadmin/templates/js/ with the following content.

$(document).ready(function(){
  $('.bxslider').bxSlider();
});

Now I add the new JS file to the websites output by adding the following the the page.includeJSlibs

page.includeJSlibs {
  slider = fileadmin/templates/js/slider.js
}

Now the page output is ready for the slider. Next I'll add a new layout in the root page TSConfig.


TCEFORM.tt_content.layout {
  addItems.51 = Bxslider
}

This adds the layout "Bxslider" to the dropdown-box "layout" for content elements.

Last you need to add some TS to the websites TS template so the new layout can be used to output the needed HTML for the slider. Add the following to your TS template


# Get default settings for images
temp.image < tt_content.image.20

# Define new layout
tt_content.image.20 = CASE
tt_content.image.20 {
  key.field = layout
  default < temp.image
  1 < temp.image
  51 < temp.image
  51 {
    imageStdWrap.dataWrap = <div class="bxslider-wrapper"><ul class="bxslider">|</ul></div>
    renderMethod = ul
    rendering {
      ul {
        oneImageStdWrap.dataWrap = <li>|</li>
      }
    }
    layout.default.value = ###IMAGES###
  }
}

# Set defaults also for TEXTPIC
tt_content.textpic.20 = < tt_content.image.20.default

That's it - the image slider is not ready to use. Create an new content element of type "images only" and add some images to it like shown on the screenshot below.


Next select in the "Appearance"-Tab the newly created layout "Bxslider" as shown below.

Open the page output and you should see the slider with the images you just selected.

Easy, is'nt it?

Notes about tt_content.stdWrap.innerWrap.cObject 
I've seen some tutorials which show, that you need to add items to tt_content.stdWrap.innerWrap.cObject with "key.field = layout" to enable the new layout. I was'nt successful with that, because it seems, that adding new items with "key.field = layout" to tt_content.stdWrap.innerWrap.cObject overrides the possibility to combine "layouts" with "Indentation and frames".

Conclusion
Please keep in mind, that this was just an easy example on how to create custom HTML output for a TYPO3 content element, so it can be used by a JQuery slider or image gallery. If you digg deeper into tt_content.image.20 TS config, you will see, that there are several other settings that can be made. Generally this all is just pure TS and you can modify it to suits the needs of the gallery or slider you wish to integrate.




Saturday, May 4, 2013

TYPO3 Extbase - own validators and multi step form validation using the old and new property mapper

TYPO3 Extbase comes with some standard validators which can be used to validate user input. When working with domain models, you can use those validators to validate the properties of the domain model.

When you use Extbase to create a form, where some part of the validation is done through an external API (e.g. logically validation) or you want to validate properties against other properties, the validation process can be more complicated. If you have a multiple step form, where all user input is collected, not persisted and finally sent to an external API which validates the input, it can be even more complicated to "jump back" to the desired step of the multiple step form and output the validation errors of the external API.

In this article I will show how to use Extbase validators to validiate a domain model and add several error messages for validation errors at once. I will also show how to display error messages for desired fields in a multiple step form after the validation process via @validate has been performed.

Besides this, the article also shows how to create a multiple step form in Extbase and how to handle validation and persistence.

At the time of writing of this article, TYPO3 6.1 with Extbase 6.1 was released. In Extbase 6.1, the new property mapper is enabled by default.

This article covers both the new and the old property mapper. I have created 2 GitHub repositories, which include all examples for the old and the new property mapper.
The code examples in this article are may be incomplete to save space. Please visit the GitHub repositories mentioned above to see the complete code.

Which property mapper to use - old or new one?

Writing this article, I spent some time with the validation classes of Extbase. Since Extbase 1.4, a lot of things like Tx_Extbase_MVC_Controller_ArgumentError or the $errors array in Tx_Extbase_Validation_Validator_AbstractValidator became deprecated and should be removed in TYPO3 6.0. Well, in TYPO3 6.0, those classed still existed and the deprecation notice now mentioned, that they will be removed in TYPO3 6.1. Some kind of confusing, I thought. Finally, as TYPO3 6.1 came out, the deprecation notice showed, that the deprecated stuff will be removed two versions after TYPO3 6.1 (so it will "survive" TYPO3 6.2 LTS). I guess, this is due to keep the backward compatibility to the upcoming LTS version of TYPO3 as high as possible.

For those who are unsure which property mapper to use, I recommend the following. If you create a new TYPO3 extension for TYPO3 6.0 or greater - use the new property mapper. It is configurable and extensible and ensures, that you don't use deprecated functions and classes.


Creating an own validator with validation errors for multiple properties

First I will show you how to create an own validator, which validates a given domain model and is able to add errors for multiple properties. The validator checks, if the given ZIP-code and city-name do match logically (e.g. for the ZIP "20095" the city name must be "Hamburg")

Assume you have a domain model "addressdata", which contains fields for a users addressdata. Straight validation like "Empty" or "Numeric" can is implemented in the domain model by using @validate annotations.

To implement the logical validation, I create a new validator in ExtBase.

Old property mapper

/**
 * Validates the given value
 *
 * @param mixed $value
 *
 * @return bool
 */
public function isValid($value) {
 $apiValidationResult = $this->apiService->validateAddressData($value);
 $success = TRUE;
 if ($apiValidationResult['zip']) {
  $error = $this->objectManager->get('Tx_Extbase_Validation_Error', $apiValidationResult['zip'], time());
  $this->errors['zip'] = $this->objectManager->get('Tx_Extbase_Validation_PropertyError', 'zip');
  $this->errors['zip']->addErrors(array($error));
  $success = FALSE;
 }
 if ($apiValidationResult['city']) {
  $error = $this->objectManager->get('Tx_Extbase_Validation_Error', $apiValidationResult['city'], time());
  $this->errors['city'] = $this->objectManager->get('Tx_Extbase_Validation_PropertyError', 'city');
  $this->errors['city']->addErrors(array($error));
  $success = FALSE;
 }
 return $success;
}

New property mapper

/**
 * Validates the given value
 *
 * @param mixed $value
 * @return bool
 */
protected function isValid($value) {
 $apiValidationResult = $this->apiService->validateAddressData($value);
 $success = TRUE;
 if ($apiValidationResult['zip']) {
  $error = $this->objectManager->get('TYPO3\\CMS\\Extbase\\Validation\\Error',
   $apiValidationResult['zip'], time());
  $this->result->forProperty('zip')->addError($error);
  $success = FALSE;
 }
 if ($apiValidationResult['city']) {
  $error = $this->objectManager->get('TYPO3\\CMS\\Extbase\\Validation\\Error',
   $apiValidationResult['city'], time());
  $this->result->forProperty('city')->addError($error);
  $success = FALSE;
 }
 return $success;
}


The code above shows, that an external API is used to validate the adress data logically. If the external API returns errors for given fields, errors are manually added for each property to the ExtBase validator.

To use the newly created validator, you just have to use the @validate annotation in your action like shown below.

Old property mapper

/**
 * Create action
 *
 * @param Tx_ValidationExamples_Domain_Model_Addressdata $newAddressdata
 * @validate $newAddressdata Tx_ValidationExamples_Validation_Validator_AddressdataValidator
 * @return void
 */
public function createAction(Tx_ValidationExamples_Domain_Model_Addressdata $newAddressdata) {
 $this->addressdataRepository->add($newAddressdata);
 $this->view->assign('message', 'Addressdata has been created');
}

New property mapper

/**
 * Create action
 *
 * @param \derhansen\ValidationExamplesNew\Domain\Model\Addressdata $newAddressdata
 * @validate $newAddressdata \derhansen\ValidationExamplesNew\Validation\Validator\AddressdataValidator
 * @return void
 */
public function createAction(\derhansen\ValidationExamplesNew\Domain\Model\Addressdata $newAddressdata) {
 $this->addressdataRepository->add($newAddressdata);
 $this->view->assign('message', 'Your new Addressdata was created.');
}

The complete source for this example has been tagged in the Github repository. Below follows direct links to the tags.

Adding custom validation errors in a multiple step form after extbase domain object validation

Sometimes the validation of a form can't be implemented by using @validate annotations. Assume you have a multiple step form, where you just collect user input and validate it using an external API in the last step of your form.

There are several approaches to create multiple step forms in Extbase. For this article I use the approach of splitting the main domain model into several small part-domain models, saving them to session variables after each step and consolidate them in the end to the main domain-object which gets persisted.

Below is a chart of the multiple step form validation process I'm going to create.



One advantage of splitting the main domain model to several small part domain models is the fact, that you can use @validation directly in the domain model without caring about the actual step of the form, where you switch validation for single properties on or off.

I created a 3 step form to enter some addressdata. The first step requires first- and lastname, the second step requires the street and streetnumber and the third step requires the zip-code and the city. When all form data has been collected, a new addressdata object is persisted to the database.

The first version of the multiple step form is tagged in the GitHub repository as example2. It includes the main validation of Extbase and is able to save the form data, when no domain model validation errors are present.
Please note, that the example2-tag for the old property mapper misses this code change, which I first discovered after the repository has been tagged.

Now I've implemented the external API service, which does some logical validation for the given address data.

/**
 * Simulates validation of addressdata entered in the multiple steps form.
 * Returns an array of validation errors for each step of the multiple steps form
 *
 * @param Tx_ValidationExamples_Domain_Model_Addressdata $addressdata
 * @return array
 */
public function validateMultipleSteps(Tx_ValidationExamples_Domain_Model_Addressdata $addressdata) {
 $errors = array();
 if ($addressdata->getStreet() == 'Elbstraße' && $addressdata->getStreetnr() > 145) {
  $errors['step2']['streetnr'] = 'Streetnr not valid for this street';
 }
 if ($addressdata->getZip() == 20095 && $addressdata->getCity() != 'Hamburg') {
  $errors['step3']['zip'] = 'ZIP Code and city do not match';
  $errors['step3']['city'] = 'ZIP Code and city do not match';
 }
 return $errors;
}

The call to the API service is implemented in the createAction() for the form. If the API service returns errors for some fields, then the createAction() saves the validation result to a session variable and redirects the user to the desired step in the multiple step form.

In the action for the given step, I've implemented a check for the validation results of the API service. This check sets validation errors to the given properties of the domain model. If there already are validation errors for the domain model, the new ones from the external API validation are added.

Old property mapper


/**
 * Sets validation errors for fields in the given step
 *
 * @param string $step The step
 * @return void
 */
protected function setApiValidationErrors($step) {
 $apiresults = $GLOBALS['TSFE']->fe_user->getKey('ses', 'apiresults');
 if (array_key_exists($step, $apiresults)) {
  /* Set Form Errors manually */
  $origErrors = $this->controllerContext->getRequest()->getErrors();
  if ($origErrors) {
   $errors = $origErrors[$step . 'data'];
  } else {
   $errors = $this->objectManager->get('Tx_Extbase_MVC_Controller_ArgumentError' ,$step . 'data');
  }

  $propertyErrors = array();

  /* Add validation errors */
  foreach ($apiresults[$step] as $key => $value) {
   $propertyErrors[$key] = $this->objectManager->get('Tx_Extbase_Validation_PropertyError', $key);
   $message = $apiresults[$step][$key];
   $propertyError = $this->objectManager->get('Tx_Extbase_Validation_Error', $message, time());
   $propertyErrors[$key]->addErrors(array($propertyError));
  }
  $errors->addErrors($propertyErrors);

  $this->controllerContext->getRequest()->setErrors(array($errors));
 }
}

New property mapper


/**
 * Sets validation errors for fields in the given step
 *
 * @param string $step The step
 * @return void
 */
protected function setApiValidationErrors($step) {
 $apiresults = $GLOBALS['TSFE']->fe_user->getKey('ses', 'apiresults');
 if (array_key_exists($step, $apiresults)) {
  /* Set Form Errors manually  - get results from property mapper and add new errors */
  $result = $this->getControllerContext()->getRequest()->getOriginalRequestMappingResults();

  /* Add validation errors */
  foreach ($apiresults[$step] as $key => $value) {
   $error = $this->objectManager->get('TYPO3\\CMS\\Extbase\\Validation\Error',
    $apiresults[$step][$key], time());
   $result->forProperty($step . 'data.' . $key)->addError($error);
  }
  $this->getControllerContext()->getRequest()->setOriginalRequestMappingResults($result);
 }
}

Since the code above is specially made for handling multiple steps / fields (like this example), below follows a more common example which shows how to set a validation error for a special property of a domain object.

Old property mapper


/* Set validation error for property */
$errors = $this->objectManager->get('Tx_Extbase_MVC_Controller_ArgumentError', 'addressdata'); 

$propertyErrors = array();

$propertyErrors['streetnr'] = $this->objectManager->get('Tx_Extbase_Validation_PropertyError', 'streetnr');
$message = 'Validation message for streetnr';
$propertyError = $this->objectManager->get('Tx_Extbase_Validation_Error', $message, time());
$propertyErrors['streetnr']->addErrors(array($propertyError));

$errors->addErrors($propertyErrors);

$this->controllerContext->getRequest()->setErrors(array($errors));

New property mapper

$result = $this->getControllerContext()->getRequest()->getOriginalRequestMappingResults();
$error = $this->objectManager->get('TYPO3\\CMS\\Extbase\\Validation\Error', 'Validation message for streetnr', time());
$result->forProperty('addressdata.streetnr')->addError($error);
$this->getControllerContext()->getRequest()->setOriginalRequestMappingResults($result);

This code adds an error for the domain object "addressdata" and sets a single validation message for the property "streetnr".

The final version of the multiple step form is tagged in the GitHub repository as example3.

Conclusion

Using the techniques shown above gives you flexibility when working with Extbase and external validation services. It also shows how to set validation errors for multiple properties at once and how to control validation results after the property mapper has processed domain validation.

As you may have noticed, the examples for the new property mapper in Extbase look more clear, contains lesser code and are better readable, since it does not use arrays to collect validation errors but objects.