Content Management System Applications within Media sites: What does this mean for testing?

With the growth of Content Management System (CMS) applications within the media sector to deliver rich content to their web applications, media companies need to consider what sort of testing is required when using CMS applications to manage rich content. The area considered here looks at integration between the external feed of data, the CMS application and the overall web platform. An example of a workflow used by a media site using CMS to manage content is illustrated below.

Basic workflow

Testing required:

• Validation to ensure that XML feed contains data required by the user.


• Tests to ensure XML feed is successfully imported into the CMS database.


• Test CMS application to ensure data with all fields are received.


• Manage content within the CMS app and verify rich content is displayed on the user facing web app.
  

There are various methods that could be considered when testing in this particular scenario. Two methods include the use of equivalence class partitioning to test a subset of the data or using automation to cross check all data. As equivalence class partitioning can always leave room for error if the different XML feeds contain different data, a better approach in this scenario would be to automate the process.

What would an automated test be required to do?

1. Parse the xml to find the feed data and cross check them against the data fields required by the user. This in turn ensures all required data is present within the feed.


2. Verify that the feed does not contain any duplicates.


3. Where images are present in the feed save all of the image URLS and collate them into a HTML file and open in a web browser to verify that there are no image URLS that produce 404 errors. This approach would be useful if there was a need to check that the correct image is displayed. However this could also be implemented programmatically without the need to launch a browser.

The journey from manual testing to automated testing using coded UI

Danny Crone, nFocus Technical Director, recently spoke at Microsoft's Connected Development conference supported by Tech.Days on 26 May in Leeds, UK.

The presentation titled "The journey from manual testing to automated testing using Coded UI" was a highly practical session which demonstrated how manual tests created in Microsoft Test Manager can be moved through fast forward play back to the end goal of full automation in Coded UI demonstrating also how data driven techniques can be applied. This session also touched upon how to play back Coded UI tests on a different browser with some general best practices.

The video of this session can be viewed below:

For other highlights from this years Tech.Days events, please click here.

Visual Studio 2010 Test Professional - Explained!

nFocus's Danny Crone meets with Giles Davies, Developer Tools Specialist from Microsoft's Developer Platform Evangelism team to talk about Visual Studio 2010 Test Professional.

Infrastructure Testing – What’s that??? Part 2

Now that you have considered the relevant areas to your particular solution, it’s time to assess which elements pose the biggest risk to you deliverable; there is no secret to how you approach this, here at nFocus we use pretty much an industry standard “Risk Based” approach.

We convene an appropriate forum of subject matter experts and elicit their thoughts on which areas pose the greatest risk, create test scenarios that will mitigate the risks identified, and test the solution, some of the areas for consideration could be:-

• Security vulnerabilities


• High Availability / Disaster Tolerance / Failover


• Connectivity and interoperability between system infrastructure


• OS Patch Status and updates


• Load balancing, Network routing and firewalls, name resolution


• System logging


• Performance


• Virtualisation


• Operational Procedures

Another list of items to consider, but at this stage you should have a pretty good idea of the shape the solution will take and the infrastructure and products that will support it. Some of the work identified during the risk assessment will be undertaken by engineers; some could be delivered as part of a contract via a hosting company for example, however you should assess what level of governance to apply to these activities; confidence in the end product for your stakeholders is paramount.

Related articles
Infrastructure Testing – What’s that??? Part 1

What mobile device/os combination should we test on?

So, at nFocus we have been doing plenty of mobile app type testing lately and I thought it would be useful and interesting to give you an insight into one particular problem that we have been hitting : What device/os combination do we test on?

I was asked recently to estimate the testing effort as part of a bid for the development of an app that would be based on the apple iOs platform. The first thing to consider was, what are the devices and os combinations that are possible. The result is shown below, there is some great information on wikipedia that helped with the collation of this. What we found really suprised me, a test would have to be run 74 times on individual device/os combinations.

Clearly, that was never going to be possible, so the question is, how do you get smart about which ones you should attempt? The answer to this is not so clear, and depends on whether there is an existing user base, from which data can be extracted. In this specific case the answer was no, so we had to take a pragmatic approach and pick some major combinations to always test on and a more exhaustive list to execute tests on toward the end of the cycle. For our purposes, we got away with testing on 3 device/os combinations with a final pass being executed on an additional 7 combinations.

One of the most important things to consider in making these decisions is to engage the customer, they must be happy with the subset that is chosen and the reasons why.

One of my major concerns when it comes to "App" testing is the perception that because we shorten the word application down to app, that the testing of these solutions does not need to be as robust : "it s just a little app, why does it cost so much to test?" Mobile applications is big business and revenues are growing from the use of such applications, we need to recognise their importance and start testing them properly :)

Infrastructure Testing – What’s That???

So you’re on a new project; a green field site and part of your remit is to test assure the infrastructure as well as the solution, where do you start? What is infrastructure testing? One definition could be “to assure that the platform the service is to be deployed on fully supports the solution in a performant and robust manner”

Considering this overview, some of the items for consideration could be:-

• Tin – Commissioning the appliances into the data centre and powering up, what should you be looking for to assure this, and weed out the DOA (Dead on Arrival) kit


• Network - Making sure that all of the Tin communicates with each other as designed, without any erroneous traffic


• Tiers – Most internet facing solutions are built using the tier model to separate the internet, application and data for security purposes, should this have a penetration test applied by a specialist in that arena


• OS – What special consideration and configuration will need to be applied to the software stack specified to support the solution? How do you assure the build prior to releasing the solution code and any supporting artefacts?


• Monitoring – What system monitoring will be in place and how do you prove its functioning to the solutions specific requirement


• Code – How can you utilise the functional testing to supply you with data for assuring the infrastructure solution

As you can see, loads of things to consider prior to any developed code being released from the CM process, this has been simplified above, the main point is that it can be planned and executed as a test discipline, utilising both testers and engineers, more to follow...

Related articles
Infrastructure Testing – What’s That??? Part 2

Software Testing Event - Last days to register!

"Getting the most out of Test within Visual Studio 2010, in the Azure Cloud!"

With projects moving to more iterative approaches like agile, why should I tightly integrate testing into the application lifecycle, and how can I use Visual Studio 2010 to help me?

There is only a few days left to register for this event, for more information, and to register your place, please visit:

http://testing-with-vs2010-in-the-azure-cloud.eventbrite.com/