Monday, December 2, 2013

Selenium tests on a Windows 8.1 machine with Internet Explorer 11

Today I tried to add an Windows 8.1 machine to my Selenium Grid and ran into some strange problems. I installed a new Windows 8.1 (32 bit) machine as a selenium node for my selenium grid. After the installation of Windows 8.1 was completed, I configured all browsers (Internet Explorer, Chrome, Firefox and Opera) and the necessary Webdrivers on the machine. Then I connected the machine to the Selenium grid and started some tests. everything seemed to run fine until I started the tests on Internet Explorer 11.

When running the tests, the browser window opened the requested URL as expected but then nothing more happened. The window did'nt even close after the test, so there must be something wrong. In the Selenium log on the machine I could see, that selenium was unable to find an element, which definitely was present on the website. After some debugging and search in the internet I found out, that Internet Explorer 11 on Windows 8.1 by default has some security settings, which must be disabled in case to run Selenium tests on the machine.

  • Disable the Protection Mode for all zones
  • Disable "Enhanced Protection Mode"

After I disabled both settings (directly in the "Internet Options") the tests were executed as expected.

Also make sure to use 32 bit versions of both the Internet Explorer driver and the operating system. I tried to get things running with the 64 bit version of Windows 8.1, but the test execution on Internet Explorer was extremely slow (it took about 4 seconds to type a character), so I ended up using the 32 bit version of Windows 8.1.

Monday, November 18, 2013

Howto debug a TYPO3 ExtBase extension with PhpStorm and XDebug


In this article I will show you step by step, how simple it is to setup PhpStorm and XDebug to debug a TYPO3 ExtBase extension. Of course you can also debug every other TYPO3 component with this setup.

Setup 

Before you can start with debugging, you need to setup XDebug, a debugging extension for you webbrowser and your PhpStorm project.

XDebug Setup
First you need to install XDebug on you system and add the following lines to your xdebug.ini (on Ubuntu 12.04 the file is located at /etc/php5/conf.d/xdebug.ini) configuration file.

xdebug.max_nesting_level = 500

xdebug.idekey = PHPSTORM
xdebug.remote_host = 127.0.0.1
xdebug.remote_enable = 1
xdebug.remote_port = 9000
xdebug.remote_handler = dbgp
xdebug.remote_mode = req

Make sure you restart your webserver after you have added updated the XDebug configuration file.

Browser Setup
Next you need to install a debug extension for you webbrowser. Since I use Google Chrome, I install the Xdebug helper, which is open source (on GitHub) and available on the Google Chrome Web Store.

PHPStorm Setup
Finally configure the server in you PhpStorm project settings like shown below.

Server Configuration

There are 2 important things to keep in mind.
  1. The host must not contain http:// 
  2. Symlinked files/directories must be mapped manually. On the screenshot above I map the typo3_src folder and the project-folder to to real location on my filesystem.
That's it - now you are ready to debug your TYPO3 ExtBase extension in PhpStorm.


Start debugging
Now it's time to start the debug session.

1. Enable your browser debug extension for the website you want to debug. In Chrome you just click on the debug icon.


2. Enable PhpStorm to listen to debug sessions by clicking on the icon shown below.


3. Set a breakpoint in your controller


4. Open the website on the server. PhpStorm now catches the incoming debug session and stops at the first breakpoint.


As you see on the screenshot above, PhpStorm stops at the breakpoint which has been set in step 3.

Happy debugging!


Troubleshooting

1. PhpStorm claims port 9000 is busy.
I had this error on an Ubuntu 12.04 development machine. Here php-fpm was installed and was listening on port 9000, so it was conflicting with XDebug. to resolve this problem I changed to port for php-fpm to listen to port 9001 instead by editing the configuration file /etc/php5/fpm/pool/www.conf

2. Debugger does'nt stop at breakpoint
When the debugger does'nt stop at the breakpoint, then it could be, that there is something wrong with the path mapping. I resolved this by adding xdebug_break(); to my code which resulted PhpStorm in stopping at the line with xdebug_break() and I could fix the path mappings at this point manually.


Tuesday, November 12, 2013

How to use content element layouts with gridelements

I often use the content element layout field in TYPO3 to add some selective CSS styling to content elements. For example, if the editor inserts a divider content element on a TYPO3 page, she/he can adjust the color of the divider by using the content element layout field like shown below.



I just define the available layouts in the page TS of the root page.

TCEFORM.tt_content.layout {
  addItems.60 = Color: Brown
  addItems.61 = Color: Gray
}

Then I use the following Typoscript to add an extra css class to the divider (e.g. brown and gray).

temp < tt_content.div
tt_content.div = CASE
tt_content.div {
  key.field = layout
  default < temp
  
  60 = COA
  60 < temp
  60.wrap = <div class="divider brown">|</div>

  61 = COA
  61 < temp
  61.wrap = <div class="divider gray">|</div>
}

Normally I globally use the defined content element layouts on other content elements too (e.g. headers), so the editor always uses the same field in TYPO3 to change the colors of content elements. You can also use the shown technique with gridelements, so the user can change the css styling of the gridelement without the need to use a flexform configuration. I just reuse the previously defined content element layout in the Typoscript of my gridelement as shown below by using a CASE on the wrap.

# ID of gridelement 
1 <  lib.gridelements.defaultGridSetup
1 {
  columns {
    # column 1
    0 < .default
    0.wrap = <div class="column1">|</div>

    # column 2
    1 < .default
    1.wrap = <div class="column2">|</div>
  }
  wrap.cObject = CASE
  wrap.cObject {
    key.field = layout
    default = TEXT
    default.value = <div class="mydefaultclass">|</div>

    60 = TEXT
    60.value = <div class="mydefaultclass brown">|</div>

    61 = TEXT
    61.value = <div class="mydefaultclass gray">|</div>
  }
}

Now the gridelement uses the TYPO3 content layout field to add an extra CSS class to its wrap.

Monday, October 28, 2013

MySQL errno: 24 after upgrade from Debian Squeeze to Wheezy

Today I upgraded one of my Debian Squeeze servers to Debian Wheezy. During the upgrade, MySQL came up with some error messages like shown below:

[ERROR] /usr/sbin/mysqld: Can't open file: './database/table1.frm' (errno: 24)
[ERROR] /usr/sbin/mysqld: Can't open file: './database/table2.frm' (errno: 24)
[ERROR] /usr/sbin/mysqld: Can't open file: './database/table3.frm' (errno: 24)
[ERROR] /usr/sbin/mysqld: Can't open file: './database/table4.frm' (errno: 24)

After some research I found out, that the cause of the problem could be, that the mysql process has too many open files. I then added the following to the my.cnf and restarted the mysql service.

[mysqld]
open_files_limit = 10000

You may adjust the limit to a value of your choice. The default limit for Debian systems is 1024, so I would recommend to set this the limit to a value greater than 1024.

After adjusting the open files limit, the mysql server was running fine again.

Tuesday, October 22, 2013

TYPO3 - PHPUnit Testing Framework tests for repositories with storagePids

When I develop a TYPO3 ExtBase extension, I often use the Testing Framework shipped with the TYPO3 extension PHPUnit. With the Testing Framework it is possible to create real testrecords in the TYPO3 database and use those records in your tests to verify everything works as expected.

In a project I faced the situation, that one of my repository methods did not find the records created with the Testing Framework. Well, after some hours of debugging I finally found the solution.

The problem

I created a new TYPO3 ExtBase extension with some tables, a controller and a view. Really nothing special about that. I then created a function in my repository which searches for some records in the database with a configurable limit. 


/**
 * Returns the latest records. 
 *
 * @param int $limit The Limit
 * @return array|\TYPO3\CMS\Extbase\Persistence\QueryResultInterface
 */
public function findLatest($limit) {
 $query = $this->createQuery();
 $query->setOrderings(array('datecreated' => QueryInterface::ORDER_DESCENDING));
 $query->setLimit((integer)$limit);
 $records = $query->execute();
 return $records;
}

Then I started to create some tests for the newly created function in the repository. One test was as following.


/**
 * Test if findLatest returns expected amout of records if limit given
 *
 * @test
 * @return void
 */
public function findLatestReturnsLimitedResult() {
 $this->testingFramework->createRecord('tx_myext_domain_model_table', array('datecreated' => time()));
 $this->testingFramework->createRecord('tx_myext_domain_model_table', array('datecreated' => time()));
 $this->testingFramework->createRecord('tx_myext_domain_model_table', array('datecreated' => time()));

 $this->createTestRecords();
 $this->assertEquals(2, $this->myRepository->findLatest(2)->count());
}

In the test above, 3 records with the Testing Framework are created and I'll use the function findLatest with a given limit of 2 to test, if really 2 records are returned.

Well, actually findLatest did return 0 records. I then created some real records in the TYPO3 backend and verified, that findLatest actually works as expected, so the problem must be somewhere in my test or in the TYPO3 setup.

So I started to debug my test and first of all, I verified, that the Testing Framework really created 3 records. Yes, 3 records were created, all having the pid 0. Ok then, since the testrecords where created correctly, there seem to be something wrong with the search. I'll then debugged the resulting SQL Query from the repository function findLatest and found out, that the records were searched in pid 1. OK, so the problem seems to have something to do with the storagePids.

ExtBase, tests and storagePid(s)

When you create an ExtBase extension, the Extension Builder automatically creates some TS for you where you can set the storagePid(s) for your extension.


persistence {
  # cat=plugin.tx_myext//a; type=string; label=Default storage PID
  storagePid =
}

This storagePid can be used to set the location of your records in the TYPO3 backend. Generally every TYPO3 Extension you create with the Extension Builder makes use of the storagePid(s) and overrides them, if a startingPoint is set in the plugin.

When you run your tests in TYPO3 backend (or through your IDE), ExtBase uses the backendConfigurationManager to get the ExtBase framework configuration. In the backendConfigurationManager class, there is a function called getDefaultStoragePid(), which returns the local function getCurrentPageId(). Let's have a look at this function.

protected function getCurrentPageId() {
 $pageId = (integer) \TYPO3\CMS\Core\Utility\GeneralUtility::_GP('id');
 if ($pageId > 0) {
  return $pageId;
 }
 // get current site root
 $rootPages = $GLOBALS['TYPO3_DB']->exec_SELECTgetRows('uid', 'pages', 'deleted=0 AND hidden=0 AND is_siteroot=1', '', '', '1');
 if (count($rootPages) > 0) {
  return $rootPages[0]['uid'];
 }
 // get root template
 $rootTemplates = $GLOBALS['TYPO3_DB']->exec_SELECTgetRows('pid', 'sys_template', 'deleted=0 AND hidden=0 AND root=1', '', '', '1');
 if (count($rootTemplates) > 0) {
  return $rootTemplates[0]['pid'];
 }
 // fallback
 return self::DEFAULT_BACKEND_STORAGE_PID;
}

So, if your TYPO3 installation where you develop your ExtBase extension has a root page, then this page ID is returned. Or if you have no root page but a TS Template which is marked as root template, then the page ID of that page is returned.

The returned page ID is used in the queryFactory class to set the storagePid(s), which leads to the situation I described earlier.

Solution / Workarounds

So how do we tell TYPO3 to search for our test records in pid 0 during tests? On solution is to set the storagePid for the ExtBase Framework to 0 like shown below.

config.tx_extbase {
  persistence {
    storagePid = 0
  }
}

This solution has some problems, since it can be overridden by other extensions and also it requires manual configuration when setting up your testing environment.

You could also just disable the check for storagePid(s) (with setRespectStoragePage(FALSE)) in your tests, but I think this solution is'nt really good, since your tests then also may find records you created in the TYPO3 backend during development.

If you want to keep your tests independent, I would recommend to set the storagePid(s) directly in the setup of the tests.

/**
 * Set up for test
 *
 * @return void
 */
public function setUp() {
 $this->testingFramework = new \Tx_Phpunit_Framework('tx_myext');
 $this->fixture = $this->objectManager->get('Derhansen\\MyExt\\Domain\\Repository\\MyRepository');

 /** @var $querySettings Typo3QuerySettings */
 $querySettings = $this->objectManager->get('TYPO3\\CMS\\Extbase\\Persistence\\Generic\\Typo3QuerySettings');
 $querySettings->setStoragePageIds(array(0));
 $this->fixture->setDefaultQuerySettings($querySettings);
}

The code above sets the storage pid to 0 and now the tests should find the records created with the testing framework and tests should not interfere with other records created manually.

Sunday, September 15, 2013

Blocking brute force attacks against the TYPO3 backend login using ModSecurity

Since the last few weeks I have noticed a constantly growing amount of failed TYPO3 backend login attempts on websites I administer. It seems I'm not alone with this problem, since a few other TYPO3 users are reporting the same issues. The method always seems to be the same - someone is doing a brute force attack against the password of the TYPO3 backend user "admin". The attack seems to be automated, takes several hours and comes from various IP addresses and from different countries in the world.

If you do not have any TYPO3 backend account with the username "admin", you could just ignore the case that someone is trying to hack your TYPO3 installation. Well, since I have configured a [BE][warning_email_addr] my mailbox gets flooded with thousands of emails about failed TYPO3 login attempts.

But what is the best way to handle and block those attacks? When I faced the first attack, I just denied the IP address of the attacker into the local .htaccess file of the TYPO3 installation. This method is implemented very quick, but always requires manual steps and is only valid for a given IP address. In the austrian TYPO3 forum I have read about a TYPO3 extension, which extends the TYPO3 backend login and is able to automatically blacklist IP addresses for failed login attempts. This idea seems to be good in general, but when you administer just more than one TYPO3 installation, the installation and configuration process of the extension can be very time consuming. I finally ended up with the solution, that once again ModSecurity seems to be the best way to handle this kind of attacks to TYPO3. During research I found an article about brute force protection using ModSecurity. Sadly the rules did'nt fit out of the box into my setup, so I had to create my own ruleset.

Below follows a ModSecurity ruleset, which blocks the IP address of an attacker for 10 minutes, if there were more than 5 failed TYPO3 authentication attempts from that IP address. Please note, that the ruleset only was tested with ModSecurity 2.7.x together with the OWASP ModSecurity Core Rule Set.

<IfModule mod_security2.c>

    #
    # TYPO3 backend login brute force protection
    #
    <Location "/typo3/index.php">
        # Turn on security engine (if disabled)
        SecRuleEngine On

        # Deny IP address if too many TYPO3 authentication failures
        SecRule IP:bf_block "@eq 1" \
        "id:'5000102', \
        msg:'IP address temporarily blocked due to too many TYPO3 authentication failures', \
        phase:2, \
        log, \
        deny"

        # Check for TYPO3 authentication failure and increment counter
        SecRule RESPONSE_BODY "LOGIN_ERROR### begin" \
        "id:'5000100', \
        msg:'Failed TYPO3 authentication attempt', \
        phase:5, \
        t:none, \
        setvar:IP.bf_counter=+1, \
        expirevar:IP.bf_counter=3600, \
        nolog, \
        pass"

        # Check for too many failures from a single IP address
        SecRule IP:bf_counter "@gt 5" \
        "id:'5000101', \
        msg:'Too many TYPO3 authentication failures from IP address', \
        phase:5, \
        t:none, \
        setvar:IP.bf_block, \
        setvar:!IP.bf_counter, \
        expirevar:IP.bf_block=600, \
        log, \
        pass"

    </Location>

</ifModule>

The ruleset creates a counter (IP.bf_counter) for each IP address, when a failed TYPO3 login is monitored. A failed login is identified by the string "LOGIN_ERROR### begin" in the response body. I don't check for the occourence of the string "Your login attempt did not succeed", since some TYPO3 installations may have a e.g. german or danish backend login and within this the resulting error message will be translated and the rule would not match.

I've decided not to log each failed login attempt, since it fills up the ModSecurity audit log with the complete response from the TYPO3 backend login. If the counter is greater than 5, the counter is unset and a new variable (IP.bf_block) with a lifetime of 600 seconds is defined. Depending on this counter, the IP address gets blocked with a 403 error.

If you have blocked your own IP address during to too many failed login attempts, you have to unset the counter variable by modifying your ruleset. This is not very practical, so another possibility to unset the variable is to simply delete the file /tmp/ip.pag - this is where ModSecurity stores all variables in a default setup. Note, that deleting the file will remove all variables set by ModSecurity. Also I don't know, if deleting the file will result in any other unwanted side-effects, so be carefull with that.

Besides using ModSecurity to protect TYPO3 websites against known and unknown attacks, I also recommend using a Yubikey to add two-factor authentication to TYPO3 admin accounts.


Wednesday, August 7, 2013

TYPO3 Error "Sorry, but an error occurred while connecting to the server." in Extension Manager

After the latest security updates for some TYPO3 extensions I was about to update those extension in a TYPO3 installation. After opening the extension manager, TYPO3 (version 4.7) did show the following error message:

Connection Problem
Sorry, but an error occurred while connecting to the server. Please check your network connection.

First I thought, there was a problem with some TYPO3 cache files, but after cleaning the typo3temp folder, the problem still occured. I then looked into to Webservers Error log and found the following logentry.

PHP Fatal error:  Cannot unset string offsets in /path/to/typo3/sysext/em/classes/tools/class.tx_em_tools.php on line 382

After some research I finally found the solution to the problem, which is the result of a bug in the TYPO3 Extension Manager, which has been fixed for some days ago.

I manually opened the file "ext_emconf.php " of every installed TYPO3 extension and checked, that the constraints-array was correct. In my case it was powermail which had a wrong entry in "ext_emconf.php".

I had to change:
'conflicts' => '',
to this:
'conflicts' => array(),

After I fixed the wrong entry, the extension manager worked fine again.

Sunday, July 21, 2013

TYPO3 ExtBase backend module with progressbar using AJAX


Assume you have created a TYPO3 ExtBase Extension and must enable the user to import data (e.g. CSV file) to the tables of your TYPO3 ExtBase extension. With ExtBase, you can easily create a TYPO3 backend module so the user can upload the CSV file and start the data import.

Now assume the user uploads a very big set of data (several 100 MBs) to be imported and starts the data import process. Well, the user will see the hourglass a long, long time and if the user is impacient, he will surely assume that something went wrong and will click somewhere else in the TYPO3 backend and within that cancel the data import.

So would'nt it be nice if you could show the user a progressbar or an updating informal text, that the import is still running?

When you look at the concepts of some extensions with backend modules in TER, you will often see, that PHP methods like flush() or ob_flush() is used to update data in a backend module for long taking processes.

With ExtBase you can't use flush(), since the output for the backend module is rendered with fluid and the output will first display, when the action that displays the content has finished.

In this article I will show how to use AJAX in a TYPO3 ExtBase backend module to show an updating JQuery UI progressbar.



Please note, that the example code located on Github has been created with TYPO3 6.1 and uses namespaces, so it only runs with TYPO3 > 6.x. But - the shown technique can also be adapted to TYPO3 4.5.

First of all, I've created a TYPO3 extension with the extension builder. The extension has a backend module called "mod1". This module uses the controller "Example" and the action "index".

/**
 * Registers a Backend Module
 */
\TYPO3\CMS\Extbase\Utility\ExtensionUtility::registerModule(
 'derhansen.' . $_EXTKEY,
 'web', // Make module a submodule of 'web'
 'mod1', // Submodule key
 '', // Position
 array(
  'Example' => 'index',
  
 ),
 array(
  'access' => 'user,group',
  'icon'   => 'EXT:' . $_EXTKEY . '/ext_icon.gif',
  'labels' => 'LLL:EXT:' . $_EXTKEY . '/Resources/Private/Language/locallang_mod1.xlf',
 )
);

The extension builder automatically creates Typoscript files with constants and setup settings for the backend module templates. Those settings look like:


templateRootPath = EXT:extbase_bemodule_ajax/Resources/Private/Backend/Templates/

As you may notice, the templates, layouts and partials are located in a subfolder named "Backend". Using this, you always need to include the extension's Typoscript in your sites Typoscript template. If you remove the folder "Backend" and just place your templates, layouts and partials in the "normal" folder-structure of an ExtBase extension, there is no need to include the Typoscript of your extension. I have done this to keep things simple.

Next I've created the layout and the template for the backend module. Notice, that I have created a viewhelper named IncludeJQueryViewHelper, which automatically includes JQuery and JQueryUi with the nesceccary CSS files to the backend module.


{namespace h=derhansen\ExtbaseBemoduleAjax\ViewHelpers\Be}

<h:IncludeJQuery jquery="1" jqueryui="1" />

<f:be.container loadExtJs="0" loadPrototype="0" loadScriptaculous="0">
...
</f:be.container>

Also notice, that the f:be.container viewhelper is configured to disable extJs, prototype and scriptaculous.

In the template for the backend module I have placed some text, a JQuery UI Progressbar and a JQuery UI Button. I also created some Javascript code, which starts starts the progressbar and also retrieves the status of the progressbar by AJAX.

When the start-button is clicked, an AJAX request (startLongProcess) is called which starts a long taking process in our ExampleController. The AJAX request is asynchronous, so the process is not blocking the TYPO3 backend. Right after the long taking process is started, another AJAX request (checkLongProcess) is called which retrieves the status of the long taking process and updates the progressbar.

To keep things simple, the long taking action in the ExampleController just executes a for-loop 20 times and does a sleep(1) on each iteration. Below follows both the startLongProcess action, which starts the long process and the checkLongProcess action, which is used to update the progressbar.


/**
 * Example for a long process (just loops 20 seconds). Returns TRUE if process is finished
 *
 * @return bool TRUE if process is finished
 */
public function startLongProcessAction() {
 for ( $i = 1; $i <= 20; $i++) {
  /* Increase counter stored in session variable */
  session_start();
  $_SESSION['progval'] = $i * 5;
  session_write_close ();
  sleep(1);
 }

 /* Reset the counter in session variable to 0 */
 session_start();
 $_SESSION['progval'] = 0;
 session_write_close ();
 return json_encode(TRUE);
}

/**
 * Checks the status of the long process
 *
 * @return int Status of process in percent
 */
public function checkLongProcessAction() {
 session_start();
 if (!isset($_SESSION['progval'])) {
  $_SESSION['progval'] = 0;
 }
 session_write_close ();
 return json_encode($_SESSION['progval']);
}

Notice, that this example uses PHP session variables to store the actual state of the long taking process. You can also use a database or other techniques to store the actual state.

When working with PHP session variables inside a method, you have to keep in mind, that the session variable is stored, after the method is finished to prevent concurrent writes to the session. To store the session variable while the for-loop is running, you have to use session_write_close() to actually store the session variable.

Finally I updated ext_tables.php so the 2 new actions can be called by the module.

Conclusion
Creating a ExtBase backend module that uses AJAX to dynamically update content on the modules page is quite simple. You can use the same techniques for backend modules as for TYPO3 frontend plugins.

The complete code for the example shown here is available on Github

Monday, July 15, 2013

Direct Mail with images as table-output in TYPO3 6.x

With TYPO3 6.x, "renderMethod = table" seem to be deprecated and does'nt seem to be fully supported anymore. For sites using Direct Mail, this could be an issue, since one of the best ways to keep HTML newsletters compatible with most of all e-mail-clients is to render the output with tables. Especially when it comes to images in newsletters, it is best to use a table layout to be sure, that the images are positioned as expected.

I had to setup a TYPO3 6.x site with Direct Mail and faced the issue described above. Also images where not displayed in the newsletter, which also seems to be an issue when you use Direct Mail in combination with TYPO3 6.x

To handle those issued the following steps are necessary:

  1. For your newsletter TS template, do not include "CSS Styled Content", but "CSS Styled Content TYPO3 v4.7". This version of CSS Styled Content still includes the table rendering method.
  2. Include config.absRefPrefix = {$plugin.tx_directmail_pi1.siteUrl} in your newsletter TS Setup. This fixes the problem with images not being displayed.
After using the settings decribed above, I could successfully send HTML e-mails using Direct Mail in TYPO3 6.x. The solution has been found using TYPO3 forge, where both issues already have been reported.


Sunday, June 9, 2013

Using TYPO3 content layouts to create Jquery image sliders or galleries

For some time ago I've write an article about how to create a JQuery Cycle image slider in TYPO3 by using the media field in TYPO3. The shown technique works fine for header sliders, but what if you need to enable the website editor to dynamically add an image gallery or slider to the website? You can  try to find a good image gallery for TYPO3 in TER or you could try to integrate the JQuery Image gallery od slider of your choise without using an extension.

In this article I will show how integrate a JQuery image slider into TYPO3 by using the layout field of TYPO3 content elements without the need to install an extension. The technique shown can be used to create custom HTML output for a lot of JQuery image galleries or sliders available. In this example I will show how to create the needed HTML output for the bxslider.

Prerequisites
You need a working TYPO3 6.1 (lower versions should work as well) installation of TYPO3 with a working template and css_styled_content installed, so TYPO3 content elements are shown on the websites output.

Integration into TYPO3
First, you need to include a version of JQuery and the image slider (both JS and CSS files) to the output of your website. Upload the JS and CSS files for the image slider to a folder in your fileadmin and add the following to your TS template

page.includeJSlibs {
  jquery = http://ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js
  jquery.external = 1
  bxslider = fileadmin/templates/js/jquery.bxslider.min.js
}

page.includeCSS {
  bxslider = fileadmin/templates/res/jquery.bxslider.css
}

After adding JQuery and the slider to your frontend output, you need to add the JS for the slider to create an instance of it for each object that should be treated as an image slider. For the bxslider, you just need some lines of code. I create the file slider.js in the directory fileadmin/templates/js/ with the following content.

$(document).ready(function(){
  $('.bxslider').bxSlider();
});

Now I add the new JS file to the websites output by adding the following the the page.includeJSlibs

page.includeJSlibs {
  slider = fileadmin/templates/js/slider.js
}

Now the page output is ready for the slider. Next I'll add a new layout in the root page TSConfig.


TCEFORM.tt_content.layout {
  addItems.51 = Bxslider
}

This adds the layout "Bxslider" to the dropdown-box "layout" for content elements.

Last you need to add some TS to the websites TS template so the new layout can be used to output the needed HTML for the slider. Add the following to your TS template


# Get default settings for images
temp.image < tt_content.image.20

# Define new layout
tt_content.image.20 = CASE
tt_content.image.20 {
  key.field = layout
  default < temp.image
  1 < temp.image
  51 < temp.image
  51 {
    imageStdWrap.dataWrap = <div class="bxslider-wrapper"><ul class="bxslider">|</ul></div>
    renderMethod = ul
    rendering {
      ul {
        oneImageStdWrap.dataWrap = <li>|</li>
      }
    }
    layout.default.value = ###IMAGES###
  }
}

# Set defaults also for TEXTPIC
tt_content.textpic.20 = < tt_content.image.20.default

That's it - the image slider is not ready to use. Create an new content element of type "images only" and add some images to it like shown on the screenshot below.


Next select in the "Appearance"-Tab the newly created layout "Bxslider" as shown below.

Open the page output and you should see the slider with the images you just selected.

Easy, is'nt it?

Notes about tt_content.stdWrap.innerWrap.cObject 
I've seen some tutorials which show, that you need to add items to tt_content.stdWrap.innerWrap.cObject with "key.field = layout" to enable the new layout. I was'nt successful with that, because it seems, that adding new items with "key.field = layout" to tt_content.stdWrap.innerWrap.cObject overrides the possibility to combine "layouts" with "Indentation and frames".

Conclusion
Please keep in mind, that this was just an easy example on how to create custom HTML output for a TYPO3 content element, so it can be used by a JQuery slider or image gallery. If you digg deeper into tt_content.image.20 TS config, you will see, that there are several other settings that can be made. Generally this all is just pure TS and you can modify it to suits the needs of the gallery or slider you wish to integrate.




Saturday, May 4, 2013

TYPO3 Extbase - own validators and multi step form validation using the old and new property mapper

TYPO3 Extbase comes with some standard validators which can be used to validate user input. When working with domain models, you can use those validators to validate the properties of the domain model.

When you use Extbase to create a form, where some part of the validation is done through an external API (e.g. logically validation) or you want to validate properties against other properties, the validation process can be more complicated. If you have a multiple step form, where all user input is collected, not persisted and finally sent to an external API which validates the input, it can be even more complicated to "jump back" to the desired step of the multiple step form and output the validation errors of the external API.

In this article I will show how to use Extbase validators to validiate a domain model and add several error messages for validation errors at once. I will also show how to display error messages for desired fields in a multiple step form after the validation process via @validate has been performed.

Besides this, the article also shows how to create a multiple step form in Extbase and how to handle validation and persistence.

At the time of writing of this article, TYPO3 6.1 with Extbase 6.1 was released. In Extbase 6.1, the new property mapper is enabled by default.

This article covers both the new and the old property mapper. I have created 2 GitHub repositories, which include all examples for the old and the new property mapper.
The code examples in this article are may be incomplete to save space. Please visit the GitHub repositories mentioned above to see the complete code.

Which property mapper to use - old or new one?

Writing this article, I spent some time with the validation classes of Extbase. Since Extbase 1.4, a lot of things like Tx_Extbase_MVC_Controller_ArgumentError or the $errors array in Tx_Extbase_Validation_Validator_AbstractValidator became deprecated and should be removed in TYPO3 6.0. Well, in TYPO3 6.0, those classed still existed and the deprecation notice now mentioned, that they will be removed in TYPO3 6.1. Some kind of confusing, I thought. Finally, as TYPO3 6.1 came out, the deprecation notice showed, that the deprecated stuff will be removed two versions after TYPO3 6.1 (so it will "survive" TYPO3 6.2 LTS). I guess, this is due to keep the backward compatibility to the upcoming LTS version of TYPO3 as high as possible.

For those who are unsure which property mapper to use, I recommend the following. If you create a new TYPO3 extension for TYPO3 6.0 or greater - use the new property mapper. It is configurable and extensible and ensures, that you don't use deprecated functions and classes.


Creating an own validator with validation errors for multiple properties

First I will show you how to create an own validator, which validates a given domain model and is able to add errors for multiple properties. The validator checks, if the given ZIP-code and city-name do match logically (e.g. for the ZIP "20095" the city name must be "Hamburg")

Assume you have a domain model "addressdata", which contains fields for a users addressdata. Straight validation like "Empty" or "Numeric" can is implemented in the domain model by using @validate annotations.

To implement the logical validation, I create a new validator in ExtBase.

Old property mapper

/**
 * Validates the given value
 *
 * @param mixed $value
 *
 * @return bool
 */
public function isValid($value) {
 $apiValidationResult = $this->apiService->validateAddressData($value);
 $success = TRUE;
 if ($apiValidationResult['zip']) {
  $error = $this->objectManager->get('Tx_Extbase_Validation_Error', $apiValidationResult['zip'], time());
  $this->errors['zip'] = $this->objectManager->get('Tx_Extbase_Validation_PropertyError', 'zip');
  $this->errors['zip']->addErrors(array($error));
  $success = FALSE;
 }
 if ($apiValidationResult['city']) {
  $error = $this->objectManager->get('Tx_Extbase_Validation_Error', $apiValidationResult['city'], time());
  $this->errors['city'] = $this->objectManager->get('Tx_Extbase_Validation_PropertyError', 'city');
  $this->errors['city']->addErrors(array($error));
  $success = FALSE;
 }
 return $success;
}

New property mapper

/**
 * Validates the given value
 *
 * @param mixed $value
 * @return bool
 */
protected function isValid($value) {
 $apiValidationResult = $this->apiService->validateAddressData($value);
 $success = TRUE;
 if ($apiValidationResult['zip']) {
  $error = $this->objectManager->get('TYPO3\\CMS\\Extbase\\Validation\\Error',
   $apiValidationResult['zip'], time());
  $this->result->forProperty('zip')->addError($error);
  $success = FALSE;
 }
 if ($apiValidationResult['city']) {
  $error = $this->objectManager->get('TYPO3\\CMS\\Extbase\\Validation\\Error',
   $apiValidationResult['city'], time());
  $this->result->forProperty('city')->addError($error);
  $success = FALSE;
 }
 return $success;
}


The code above shows, that an external API is used to validate the adress data logically. If the external API returns errors for given fields, errors are manually added for each property to the ExtBase validator.

To use the newly created validator, you just have to use the @validate annotation in your action like shown below.

Old property mapper

/**
 * Create action
 *
 * @param Tx_ValidationExamples_Domain_Model_Addressdata $newAddressdata
 * @validate $newAddressdata Tx_ValidationExamples_Validation_Validator_AddressdataValidator
 * @return void
 */
public function createAction(Tx_ValidationExamples_Domain_Model_Addressdata $newAddressdata) {
 $this->addressdataRepository->add($newAddressdata);
 $this->view->assign('message', 'Addressdata has been created');
}

New property mapper

/**
 * Create action
 *
 * @param \derhansen\ValidationExamplesNew\Domain\Model\Addressdata $newAddressdata
 * @validate $newAddressdata \derhansen\ValidationExamplesNew\Validation\Validator\AddressdataValidator
 * @return void
 */
public function createAction(\derhansen\ValidationExamplesNew\Domain\Model\Addressdata $newAddressdata) {
 $this->addressdataRepository->add($newAddressdata);
 $this->view->assign('message', 'Your new Addressdata was created.');
}

The complete source for this example has been tagged in the Github repository. Below follows direct links to the tags.

Adding custom validation errors in a multiple step form after extbase domain object validation

Sometimes the validation of a form can't be implemented by using @validate annotations. Assume you have a multiple step form, where you just collect user input and validate it using an external API in the last step of your form.

There are several approaches to create multiple step forms in Extbase. For this article I use the approach of splitting the main domain model into several small part-domain models, saving them to session variables after each step and consolidate them in the end to the main domain-object which gets persisted.

Below is a chart of the multiple step form validation process I'm going to create.



One advantage of splitting the main domain model to several small part domain models is the fact, that you can use @validation directly in the domain model without caring about the actual step of the form, where you switch validation for single properties on or off.

I created a 3 step form to enter some addressdata. The first step requires first- and lastname, the second step requires the street and streetnumber and the third step requires the zip-code and the city. When all form data has been collected, a new addressdata object is persisted to the database.

The first version of the multiple step form is tagged in the GitHub repository as example2. It includes the main validation of Extbase and is able to save the form data, when no domain model validation errors are present.
Please note, that the example2-tag for the old property mapper misses this code change, which I first discovered after the repository has been tagged.

Now I've implemented the external API service, which does some logical validation for the given address data.

/**
 * Simulates validation of addressdata entered in the multiple steps form.
 * Returns an array of validation errors for each step of the multiple steps form
 *
 * @param Tx_ValidationExamples_Domain_Model_Addressdata $addressdata
 * @return array
 */
public function validateMultipleSteps(Tx_ValidationExamples_Domain_Model_Addressdata $addressdata) {
 $errors = array();
 if ($addressdata->getStreet() == 'Elbstraße' && $addressdata->getStreetnr() > 145) {
  $errors['step2']['streetnr'] = 'Streetnr not valid for this street';
 }
 if ($addressdata->getZip() == 20095 && $addressdata->getCity() != 'Hamburg') {
  $errors['step3']['zip'] = 'ZIP Code and city do not match';
  $errors['step3']['city'] = 'ZIP Code and city do not match';
 }
 return $errors;
}

The call to the API service is implemented in the createAction() for the form. If the API service returns errors for some fields, then the createAction() saves the validation result to a session variable and redirects the user to the desired step in the multiple step form.

In the action for the given step, I've implemented a check for the validation results of the API service. This check sets validation errors to the given properties of the domain model. If there already are validation errors for the domain model, the new ones from the external API validation are added.

Old property mapper


/**
 * Sets validation errors for fields in the given step
 *
 * @param string $step The step
 * @return void
 */
protected function setApiValidationErrors($step) {
 $apiresults = $GLOBALS['TSFE']->fe_user->getKey('ses', 'apiresults');
 if (array_key_exists($step, $apiresults)) {
  /* Set Form Errors manually */
  $origErrors = $this->controllerContext->getRequest()->getErrors();
  if ($origErrors) {
   $errors = $origErrors[$step . 'data'];
  } else {
   $errors = $this->objectManager->get('Tx_Extbase_MVC_Controller_ArgumentError' ,$step . 'data');
  }

  $propertyErrors = array();

  /* Add validation errors */
  foreach ($apiresults[$step] as $key => $value) {
   $propertyErrors[$key] = $this->objectManager->get('Tx_Extbase_Validation_PropertyError', $key);
   $message = $apiresults[$step][$key];
   $propertyError = $this->objectManager->get('Tx_Extbase_Validation_Error', $message, time());
   $propertyErrors[$key]->addErrors(array($propertyError));
  }
  $errors->addErrors($propertyErrors);

  $this->controllerContext->getRequest()->setErrors(array($errors));
 }
}

New property mapper


/**
 * Sets validation errors for fields in the given step
 *
 * @param string $step The step
 * @return void
 */
protected function setApiValidationErrors($step) {
 $apiresults = $GLOBALS['TSFE']->fe_user->getKey('ses', 'apiresults');
 if (array_key_exists($step, $apiresults)) {
  /* Set Form Errors manually  - get results from property mapper and add new errors */
  $result = $this->getControllerContext()->getRequest()->getOriginalRequestMappingResults();

  /* Add validation errors */
  foreach ($apiresults[$step] as $key => $value) {
   $error = $this->objectManager->get('TYPO3\\CMS\\Extbase\\Validation\Error',
    $apiresults[$step][$key], time());
   $result->forProperty($step . 'data.' . $key)->addError($error);
  }
  $this->getControllerContext()->getRequest()->setOriginalRequestMappingResults($result);
 }
}

Since the code above is specially made for handling multiple steps / fields (like this example), below follows a more common example which shows how to set a validation error for a special property of a domain object.

Old property mapper


/* Set validation error for property */
$errors = $this->objectManager->get('Tx_Extbase_MVC_Controller_ArgumentError', 'addressdata'); 

$propertyErrors = array();

$propertyErrors['streetnr'] = $this->objectManager->get('Tx_Extbase_Validation_PropertyError', 'streetnr');
$message = 'Validation message for streetnr';
$propertyError = $this->objectManager->get('Tx_Extbase_Validation_Error', $message, time());
$propertyErrors['streetnr']->addErrors(array($propertyError));

$errors->addErrors($propertyErrors);

$this->controllerContext->getRequest()->setErrors(array($errors));

New property mapper

$result = $this->getControllerContext()->getRequest()->getOriginalRequestMappingResults();
$error = $this->objectManager->get('TYPO3\\CMS\\Extbase\\Validation\Error', 'Validation message for streetnr', time());
$result->forProperty('addressdata.streetnr')->addError($error);
$this->getControllerContext()->getRequest()->setOriginalRequestMappingResults($result);

This code adds an error for the domain object "addressdata" and sets a single validation message for the property "streetnr".

The final version of the multiple step form is tagged in the GitHub repository as example3.

Conclusion

Using the techniques shown above gives you flexibility when working with Extbase and external validation services. It also shows how to set validation errors for multiple properties at once and how to control validation results after the property mapper has processed domain validation.

As you may have noticed, the examples for the new property mapper in Extbase look more clear, contains lesser code and are better readable, since it does not use arrays to collect validation errors but objects.


Tuesday, April 23, 2013

Asynchronous banner management for TYPO3

Today I published my banner management extension sf_banners for TYPO3. It is able to show image, flash and HTML banners on a TYPO3 website. Each banner contains a statistic which shows the banners' impressions and clicks. The banners can be limited to an maximum amount of clicks/impressions and you can control the date on which the banners are shown by using the TYPO3 start- and stop function.


One of the main features of the extension is the ability to load banners asynchronously, so the main load time af a page is not being affected by the plugins rendering process.


The extension has been developed using Extbase and Fluid to ensure compatibility to existing and future TYPO3 versions. Also test driven development was one of the main aspects in the development process to ensure, that new features and code changes do not break existing functionality. The extension has been tested with all current versions of TYPO3 from 4.5.x to 6.0.x

If you have any comments or suggestions, feel free to post them on the extensions GitHub projectpage.

Development process

Below follows a short summary about the development process of the extension.

The first approach of the extension was to use pure Extbase/Fluid functionality to display the banners on a page. Since the extension has to update impressions and also has to select the banners from the repository on each request, the plugin has to be non-cached. After having finished this approach, I was disappointed by the performance of the page rendering time. As I tested the extension on several TYPO3 versions, I also found out, that the page rendering time decreased the higher the TYPO3 version got.

So I read a lot of discussions on the TYPO3 mailinglists and the internet about Extbase and performance tuning and ended up with the solution to load all banners asynchronously by Ajax. To do so, I implemented JQuery and added an action to the controller which delivered the banners content as pure HTML. After that, everything seemed to run fine. Since the plugin now was cached and content fetched by Ajax, the page load time was'nt mainly affected by the extension. Sadly my integration tests showed, that there were problems with HTML/JS banners containing document.write JavaScript code. As the banners HTML/JS code was loaded asynchronously, a document.write statement would be executed after the page DOM is loaded. The result was, that the main page was loaded and shortly after that, the HTML/JS banner was shown and the formerly rendered page output was completely removed.

The solution to get rid of the document.write problem was postscribe. This standalone JS library overrides document.write and enables the usage of document.write after the DOM is ready.

All development steps are filed on GitHub and you can look at each of the approaches described above, as the different steps have been tagged.


Saturday, March 16, 2013

Setting up Jenkins CI, Selenium grid and PHPUnit Selenium to perform integration testing

When developing bigger TYPO3 websites with a lot of own extensions and complex functionality, it is nescessary to make sure, that the integration of new features or changes to existing code does not break the functionality of the website.

To meet those requirements, it is recommended to use a continuous integration platform (like Jenkins CI) to automate the testing of the software. The platform can be used to automatically run the unit tests of your TYPO3 ExtBase extensions. If you use the testing framework included in the TYPO3 extension "phpunit", the tests can also help you testing your code with real data from the database.

Another thing you can (and should) test is the website and it's functionality itself. To do so, you can use a browser automation tool to run automated tests against real browsers. One tool available is the Selenium Server.

In this article I will show you how to build up a continuous integration solution consiting of Jenkins CI, Selenium Grid and 3 nodes with different webbrowsers and operating systems. Finally I will show you how to create some simple selenium tests with PHPUnit selenium and show, how to automatically run them against different environment configurations. Please note, that I will not go into deep details installing and configuring each software.

Setting up Jenkins CI and Selenium Grid

First of all, you need to install Jenkins CI on a server. For this article, a fresh Debian Server has been installed with all dependencies for Jenkins CI and finally, the latest release of Jenkins CI has been downloaded from the Jenkins CI homepage. Installation and configuration is really simple as described here. After installation and configuration, Jenkins CI is ready for use. To run PHPUnit tests, make sure you have installed PHPUnit, PHPUnit Selenium and all necessary extensions you need to run your tests.

The next step is to install and configure Selenium on the Jenkins CI server. This instance of Selenium will run Selenium Grid, which will act as the master Selenium grid hub, where all other Selenium nodes connect to. Open the Jenkins Plugin manager and install the Selenium plugin. Installation is really easy and you just have to install the plugin and restart your Jenkins CI. In the Jenkins CI web interface you now have the new menu "Selenium grid" as shown on the screenshot below.


Selenium Grid is ready to handle nodes. Now it is time to configure the Selenium nodes.

Configuring the Selenium nodes

For this article, 3 virtual machines with different browser configurations have been created and configured.

  • Windows XP 32 bit with the following webbrowsers
    • Internet Explorer 8
    • Firefox 19
    • Opera 12
    • Chrome 25
  • Windows 7 32 bit with the following webbrowsers
    • Internet Explorer 9
    • Firefox 19
    • Opera 12
    • Chrome 25
  • Ubuntu 12.10 64 bit with the following webbrowsers
    • Firefox 19
    • Chrome 25

For all virtual machines, JAVA and the browsers listed above has been installed. Chrome, Opera and Internet Explorer requires drivers, so the locally installed Selenium Server can remote control the local webbrowser. Those drivers can be downloaded here or directly via the Selenium homepage. Finally, the latest release of  Selenium has been downloaded to the machines.

For all nodes, the following command has been used to start Selenium.


java -jar \path\to\selenium-server-standalone-2.29.0.jar -role node -hub http://jenkins-server:4444/grid/register -browser "browserName=internet explorer,version=8,maxInstances=1" -browser "browserName=firefox,version=19,maxInstances=5" -browser "browserName=opera,version=12,maxInstances=5" -browser "browserName=chrome,version=25,maxInstances=5"




The node connects to the master Selenium hub and registers the local browsers to the hub. Note, that the parameter "-browser" does not accept spaces in between the individual arguments.

After all nodes have been configured and the local Selenium Server has been started, the Selenium Grid plugin in Jenkins CI should show all registered nodes as shown on the screenshot below.


The Selenium Grid and all nodes are now ready for usage.

The first Selenium tests

Now you are ready to write the first test. Create a new PHP file (selenium-tests.php) and include the following contents.

<?php
class DefaultTest extends PHPUnit_Extensions_Selenium2TestCase {

    /*
     * Setup
     */
    protected function setUp() {
        $this->setHost('your-jenkins');
        $this->setBrowser('internet explorer');
        $this->setBrowserUrl('http://www.google.com');
    }

    /**
     * @test
     */
    public function testTitle() {
        $this->url('http://www.google.com');
        $this->assertEquals('Google', $this->title());
    }
}
?>

Use the setHost() method to set the hostname of your jenkins. This can be an IP-Address or a hostname. Do not include any ports - PHPUnit Seleniums does this for you.

Before setting up a job in jenkins (which I do not show in this article), you can run the test locally by using the following command.

phpunit selenium-tests.php

Now the test is sent to the Selenium Grid server and executed an a node matching the desired browser.

The result should look like shown below

PHPUnit 3.7.13 by Sebastian Bergmann.

.

Time: 6 seconds, Memory: 6.00Mb

OK (1 test, 1 assertion)

Now we want to execute the test on a node with Internet Explorer version 8. To do so, we need to set the desired capabilities of the test. Modify the file selenium-tests.php to the following:

<?php
class DefaultTest extends PHPUnit_Extensions_Selenium2TestCase {

    /*
     * Setup
     */
    protected function setUp() {
        $this->setHost('your-jenkins');
        $this->setDesiredCapabilities( array('version' => '8') );
        $this->setBrowser('internet explorer');
        $this->setBrowserUrl('http://www.google.com');
    }

    /**
     * @test
     */
    public function testTitle() {
        $this->url('http://www.google.com');
        $this->assertEquals('Google', $this->title());
    }
}
?>

Now run the test again and you should see on the Windows XP machine, that Internet Explorer 8 is launched and the URL www.google.com is opened.

That's all. You can modify the desired capabilities to suit the needs of your tests. Since the parameter accepts an array, you can also set the browser version and the target platform together.

The last step is to check in your tests to a repository (e.g. GIT) and setup a new job in Jenkins to execute the selenium tests periodically.

Saturday, March 2, 2013

Integrating a JQuery Cycle image slider in TYPO3 6.0 without the need of installing an extension

There are a lot of JQuery image slider or slideshow extensions in TYPO3 TER, which (hopefully) all do work very well. Integration is mostly very easy - just install the extension, include some static TS and configure the plugin and there you go with a nice "Out of the box" Image Slider. Thit is surely good for a lot of people using TYPO3.

But some extensions come with their own included version of JQuery, which can't be disabled. Others include a lot of inline CSS and/or JS into the frontend, which also can't be disabled. And some extensions do not work correctly with TYPO3 6.0.

On the way looking for the "perfect" solution for a JQuery Image Slider in TYPO3, I decided not to use an extension (no worry about extension updates, breaking changes or incompatibility), but integration the slider directly into the TYPO3 website using default and well known TYPO3 techniques. The advantages of this is: full flexibility, easy administration and easy usage for editors.

This article shows how to create a JQuery image slider using Jquery Cycle, which is generated from a normal TYPO3 image content element. The slider includes a bullet navigation depending on the number of images.

Prerequisites
You need a working TYPO3 6.0 (lower versions should work as well) installation with CSS Styled Content installed and at least one template, so you actually are able to insert content on a page and see it's output in the frontend.

Integration into TYPO3
First of all, you need download the JQuery Cycle Plugin and upload it somewhere in your fileadmin directory (in this example fileadmin/templates/js/jquery.cycle.all.js).

Next, create a new JS file (fileadmin/templates/js/slider.js), where you put the JS for the JQuery Cycle slider. Insert the following content to the file.

$(document).ready(function () {
    $(".imageslider .csc-textpic-imagewrap").cycle({
        fx:'fade',
        pause:1,
        pager:'.slidernav',
        pagerAnchorBuilder:function paginate(idx, el) {
            return '<a class="' + idx + '" href="#" >•</a>';
        }
    });
});

Now, include the JQuery Cycle Plugin and the newly created JS file to your template. If you have not already included a version of JQuery to your site, make sure to include one.

page.includeJS {
  jquery = http://ajax.googleapis.com/ajax/libs/jquery/1.8.3/jquery.min.js
  jquery.external = 1
  cycle = fileadmin/templates/js/jquery.cycle.all.js
  slider = fileadmin/templates/js/slider.js 
}

The slider also requires some CSS for positioning of the bullet navigation. Include the following CSS to your sites CSS file.

.imageslider {
    position: relative;
    width: 300px;
    height: 225px;
    overflow: hidden;
}

.slidernav {
    position: absolute;
    z-index: 10;
    top: 170px;
    right: 20px;
}

.slidernav a {
    text-decoration: none;
    color: #ffffff;
    font-size: 40px;
    margin: 0 0 0 5px;
}

.slidernav a.activeSlide {
    color: #bbbbbb;
}

/* remove default bottom margin */
.imageslider .csc-textpic-image {
  margin: 0;
}

Please notice, that in this example I set the width and height of the image slider to a width of 300 pixel and a height of 225 pixel. This is'nt really necessary, but it makes it easier to position the bullet navigation to the bottom right.

As I don't like to override the default settings of CSS styled content or tt_content, I create a new frame, which the editor can select in the frame-dropdown of each content element.

Include the following to your root Page TSConfig.

TCEFORM.tt_content.section_frame {
     addItems.100 = Slider 
}

Now you must enable the new frame and include the HTML tags for the slider with the following TS.

tt_content.stdWrap.innerWrap.cObject {
  100 = TEXT
  100.value = <div class="imageslider"><ul class="slidernav"></ul>|</div>
}

That's all - the JQuery Cycle image slider is now ready for usage.

Usage
Create a new content element with images only and insert some images.


Set the "Indentation and Frames" to "Slider", the width of the images and the image alignment to 1 column.

Save the content element and you're done.

If you open the page in the frontend, you should see an image slider like shown on the screenshot below.


Conclusion
I hope this article gives you a perspective on what is possible with TYPO3's image content element and just some lines of TS, JS and CSS. If you want a next and previous button for the slider, no problem - just use the "prev" and "next" option of JQuery Cycle plugin.

As you have seen, it is not always necessary to install an extension to integrate an image slider to TYPO3. With the shown solution, editors can use TYPO3's default content elements and the site administrator has full control of the sliders features and can use all nice options of the JQuery Cycle plugin.

Please notice, that everything shown in this article is just an example on how a JQuery Cycle image slider could be integrated into TYPO3. Feel free to modify the settings to your own needs.

Update 15.05.2013
You have to make sure that the JS file, which enables the JQuery Cycle slider, gets the DIV-tag, which contain the container elements for the images. Below is an example for images, which are aligned "above, center". The second line contains the part, where you select the DIV with elements to cycle.

$(document).ready(function () {
    $(".imageslider .csc-textpic-center-inner").cycle({
        fx:'fade',
        pause:1,
        pager:'.slidernav',
        pagerAnchorBuilder:function paginate(idx, el) {
            return '';
        }
    });
});

Saturday, February 23, 2013

Using FAL media in TYPO3 6.0 for inline CSS header images

Last year I wrote an article about how to use a page resources to create an "sliding" header-image, which is included by inline CSS.

In TYPO3 6.0 the FAL (File Abstraction Layer) was introduced, which changes a lot of things regarding file handling. The technique I described in the article mentioned before does not work with TYPO3 6.0, since page resources now are handled by FAL.

In this article I will describe how to create a sliding header-image in TYPO3 6.0 using FAL and the resources of a page.

First you should upload an image which should be uses as the header image. Just upload it somewhere in fileadmin.

Next you select the page, where the header image should be shown. Create a new relation to the formerly uploaded file.



Now you must add the following TS snippet to your TypoScript template

page.cssInline {
  10 = FILES
  10 { 
     references.data =  levelmedia:-1, slide
     references.listNum = 0
     renderObj = TEXT
     renderObj.data = file:current:publicUrl
     renderObj.wrap (
      .header {
        background-image: url(../|);  
      }
    ) 
  }  
}

Please notice, that the snippet above just is an example which uses the first file (listNum = 0) from the resources of the page.

The result is a new inline CSS stylesheet in the frontend of your website, where the formerly uploaded image is used as a background-image for the class "header". Below is the content of the CSS file.

.header {
        background-image: url(../fileadmin/images/typo3-logo.png);  
      }

Sunday, February 17, 2013

Using an Extbase extension in Typoscript

Using typoscript to insert or configure an Extbase extension is quite easy. I use the following typoscript code to assign an Extbase extension to a typoscript object.


myObject = USER
myObject {
     userFunc = tx_extbase_core_bootstrap->run
     extensionName = YourExtensionName
     pluginName = YourPluginName
     switchableControllerActions {
       YourControllerName {
         1 = youraction
       }
     }
     settings =< plugin.tx_yourextensionname.settings
     persistence =< plugin.tx_yourextensionname.persistence
     view =< plugin.tx_yourextensionname.view
}

The first thing to notice is, that the name of your Extension must be in UpperCamelCase. so if your extension key is "your_extension_name", then you have to set it to "YourExtensionName". This also applies to the plugin name.

The next thing you have to do is to configure the section switchableControllerActions with the controller- and the action-name you wish to use.

Last but not least you should assign the settings, the persistence- and the view-configuration to the typoscript object. Now you`re ready to use your Extbase extension in typoscript.

Notice in the example above, that the extension`s typoscript settings are imported to the new typoscript object. With this, you can modify the settings for the typoscript object individually. The same applies to the persistence- and the view-configuration. With this, you are very flexible using your Extbase extension in typoscript.

Friday, January 25, 2013

Configuring Jenkins CI to use multiple TYPO3 versions as build target for Extbase extensions


If you develop TYPO3 extbase extensions, you should know the concepts of test driven development (TDD). Covering as much as possible of your code with tests ensures, that new code or code changes don't break the functionality of your extension. In bigger projects, where several developers work together on an Extbase extension, it is helpfull to use a continuous integration server to automate testing and to ensure code quality.

If the Extbase extension must be compatible with different TYPO3 versions, you have to ensure this during the development process. In this situation, it can be helpful to run automated tests with different TYPO3 versions as build target.

In this article, I will show how to configure a single job in Jenkins CI to use multiple TYPO3 versions as a build target for  running Extbase extension tests.

Before you can start to configure the job in Jenkins CI, you must configure different TYPO3 installations on the server where Jenkins CI is located. This is necessary, because running phpunit tests against an Extbase extension requires the extension to be installed on an working TYPO3 installation.

Configuring the TYPO3 installations

TYPO3 basic setup
On the Jenkins CI Server, you create 3 new virtual hosts. Each virtual host contains a TYPO3 installation with a different version of TYPO3 (4.5.x, 4.7.x and 6.0.x). To keep the TYPO3 installation maintainable, you can symlink each TYPO3 source to a central place on the server, so you easily can update TYPO3 build versions (e.g. version 4.5.20 to 4.5.21)

After the TYPO3 setup, each TYPO3 installation can be opened locally by a hostname like the following
  • http://typo3-45.build.jenkins.local/
  • http://typo3-47.build.jenkins.local/
  • http://typo3-60.build.jenkins.local/
Setting permissions
Next you have to login to each TYPO3 installation and update some settings in the install tool.

Set the file mode mask to:
[BE][fileCreateMask] = 0666 
[BE][folderCreateMask] = 0777

This is necessary for TYPO3 6.0, because there seems to be some problems with local permissions in typo3temp/ (Uncaught TYPO3 Exception: #1294586098: Lock file could not be created) when running the tests through Jenkins CI. 

Installing and configuring the TYPO3 extension "phpunit"
In order to run automated tests in a TYPO3 installation, you have to install the extension "phpunit" from TER. Be sure to get a compatible version of phpunit for each TYPO3 version you use. 

After the extension "phpunit" has been installed, you must create a backend user named "_cli_phpunit". Just create the user and set a random password. 

Make sure, that you can at least execute the tests which came with "phpunit" successfully. This step only ensures, that tests actually can be run successfully.

Setting up the job in Jenkins

The next step is to configure the job in Jenkins. First of all, we create an empty job in Jenkins.



The next step is to create an ANT Build File (built.xml), which contains all settings for  the job. Below is an example ANT Build File for the job described in this article.

<project name="TYPO3-Extbase-Extension" default="build" basedir=".">
        <property name="output" location="${basedir}/build/"/>
        <property file="build.properties" />

        <target name="init">
                <mkdir dir="${output}"/>
                <mkdir dir="${output}/phpcs/"/>
        </target>

        <target name="build" depends="init, typo3-45-unittests, typo3-47-unittests, typo3-60-unittests, phpcs, phpmd, phpcpd">
        </target>

        <target name="typo3-45-unittests">
                <exec executable="php" failonerror="true">
                        <arg path="${typo3path-45}/typo3/cli_dispatch.phpsh" />
                        <arg value="phpunit" />
                        <arg path="${basedir}/src/Tests" />
                </exec>
        </target>

        <target name="typo3-47-unittests">
                <exec executable="php" failonerror="true">
                        <arg path="${typo3path-47}/typo3/cli_dispatch.phpsh" />
                        <arg value="phpunit" />
                        <arg path="${basedir}/src/Tests" />
                </exec>
        </target>

        <target name="typo3-60-unittests">
                <exec executable="php" failonerror="true">
                        <arg path="${typo3path-60}/typo3/cli_dispatch.phpsh" />
                        <arg value="phpunit" />
                        <arg path="${basedir}/src/Tests" />
                </exec>
        </target>

        <target name="phpcs">
                <exec executable="phpcs">
                        <arg line="--report=checkstyle
                                --report-file=${output}/phpcs/checkstyle.xml
                                --standard=/var/lib/jenkins/external_libraries/PHP_CodeSniffer/TYPO3/ruleset.xml
                                ${basedir}" />
                </exec>
        </target>

        <target name="phpmd">
                <exec executable="phpmd">
                        <arg line=" . xml codesize,unusedcode,naming,design --reportfile ${output}/messdetector.xml --exclude Tests/" />
                </exec>
        </target>

        <target name="phpcpd">
                <exec executable="phpcpd">
                        <arg line=" --log-pmd ${output}/phpcpd.xml ." />
                </exec>
        </target>
</project>

The ANT Build File also requires a configuration file (built.properties) with some properties.

php=/usr/bin/php5
typo3path-45=/var/www/path-to-your-typo3-45-root
typo3path-47=/var/www/path-to-your-typo3-47-root
typo3path-60=/var/www/path-to-your-typo3-60-root

After both files have been created, copy them to the /workspace directory of your job. At his point, the workspace directory should not exist and you have to create it manually. Make sure you set the correct permissions, so the jenkins user has read-write permissions to that directory.

The next thing you have to do is to configure the repository, where Jenkins CI checks out the TYPO3 Extbase extension. In this article, the Extbase extension is located in a GIT Repository. Configure the Repository URL, the Branch and set the "Local subdirectory for repo (optional)" to "src". Below is a screenshot of the settings.



Now you have to configure the ANT Build File. Just insert the path to the Build File as shown below.



The ANT Build File above also contains some settings for phpcs, phpmd and phpcpd. If you want to use Checkstyle, PHP messdetection and PHP duplicate code detection, you can configure those settings in the Post-Build Actions as shown on the screenshot below.



Now your job is ready to run the first time. Actually it will fail, since the configuration process is not finished yet, but it is required to run the job, so the Extbase extension is checked out from the GIT repository.

Installing the Extbase extension and running the job

After running the job the first time, you should have the folder /src inside the jobs workspace directory. This folder contains the extension, which we now symlink to the /typo3conf/ext directory of each TYPO3 installation we created earlier. Make sure, that the name of the symlink is identical to the extension key.

The symlink ensures, that updates to the extension automatically are available in all TYPO3 installations.

Now you have to login to each TYPO3 installation and install the Extbase extension with the extension manager.

When the Extbase extension is installed, you are ready to run the job and will (hopefully) see, that your Extbase extension´s tests will run on all configured TYPO3 versions.



You can also configure the job only to use a special TYPO3 version as build target as shown below.

The above example will only run tests against TYPO3 version 4.5.

Additional notes

During the development process of an extension, the table-structure may change due to new or changed fields. Those changes are not automatically updated in the local TYPO3 installations, where the extension is installed, so you have to make sure, that you update the extension´s table structure by using the extension manager in each TYPO3 installation, if you made changes to the extension´s table structure.



Thursday, January 3, 2013

Logwatch filter for ModSecurity 2

I often use ModSecurity 2 and the OWASP ModSecurity Core Rule Set (CRS) to protect a website from potential attacks. ModSecurity 2 is able to write blocked attacks to a audit logfile, so you actually can see, which Core Rule and which data matched the attack that has been blocked. The logfile can also help you to analyze false positives, so you can modify the CRS to your needs.

As a server admin, you regulary should check the logfiles of your server. One tool to help you analyzing your server´s logfiles is Logwatch, which can send reports by e-mail with a summary of the logfile analysis. Sadly Logwatch was´nt able to analyze ModSecurity 2 audit logfiles and I could´nt find a filter for Logwatch, which fullfilled my needs.

So I wrote a filter for Logwatch, which analyzes a ModSecurity 2 audit logfile for blocked attacks and collects those information for a given time period as a report. The report is seperated by vhost, so you can have a quick overview on which attacks have been blocked on which vhost. Also the reports contains a top 10 summary of blocked IP addresses.

Here is a sample output from the filter:
--------------------- ModSecurity2 (mod_security2) Begin ------------------------

ATTACKS BLOCKED ON VHOSTS:

subdomain.domain.tld - 2 time(s)
[ip: xxx.xxx.xxx.xxx] [id: 981231 ] [msg: SQL Comment Sequence Detected.]  - 1 time(s)
[ip: xxx.xxx.xxx.xxx] [id: 981231 ] [msg: SQL Comment Sequence Detected.]  - 1 time(s)

www.site.tld - 1 time(s)
[ip: xxx.xxx.xxx.xxx] [id: 990012 ] [msg: Rogue web site crawler]  - 1 time(s)
[ip: xxx.xxx.xxx.xx] [id: 981318 ] [msg: SQL Injection Attack: Common Injection Testing Detected]  - 5 time(s)
[ip: xxx.xxx.xxx.xx] [id: 950901 ] [msg: SQL Injection Attack: SQL Tautology Detected.]  - 2 time(s)

www.anothersite.tld - 1 time(s)
[ip: xxx.xxx.xxx.xxx] [id: 958291 ] [msg: Range: field exists and begins with 0.]  - 1 time(s)

TOP 10 BLOCKED IPS:
xxx.xxx.xxx.xxx - 2 time(s)
xx.xxx.xxx.xxx - 1 time(s)
xxx.xxx.xx.xx - 1 time(s)
xxx.xxx.xxx.xx - 1 time(s)
xxx.xxx.xxx.xxx - 1 time(s)

---------------------- ModSecurity2 (mod_security2) End -------------------------

The filter has been tested with ModSecurity 2 version 2.6.0 (CRS 2.2.0) and version 2.7.1 (CRS 2.2.6)

I published the Logwatch filter for Mod Security 2 on Github, so feel free to submit change requests or bug reports.