"In Lebanon, figures are just an opinion and studies are not always trusted," stated George Jabbour President of the Advertising Association (AA), as he addressed the audience that was invited at the Hilton Habtoor Hotel to be introduced to the Lebanese Media Establishment Survey that was sponsored by the syndicate. A TAM audit in 2015 on both Ipsos Nielsen & and GFK/ARA led to a serious debate on the imperative of an Establishment Survey (ES). A tender process included seven research firms (Ara, Parc, CASN, Statistics Lebanon, Sofres, Reach Mass, & Information International). Reach Mass won the tender, and started the work in Autumn 2018. ArabAd caught up with Reach Mass Chairman and Managing Director, Joseph Khoury, to learn more about this massive survey, which everyone in the industry describes as a major achievement.
Let's start from the beginning: what prompted the Advertising Association to conduct an ES?
The AA has commissioned an Establishment Survey. The aim is to provide a reliable and unique sampling base for all media measurements through a common description of socio-demographic variables. This is a mandatory step to conduct cross media comparisons through ensuring research suppliers are using same universe and same population profiles.
The ES is designed to provide a demographic, geographic, lifestyle, product and multi-media landscape of Lebanon. It is used by media owners, marketers, and advertisers as a strategic inter-media planning tool, as the ES surveys reflect media usage across all media types by all major demographics.
ES also serves to provide the research company with a database of households, from which a panel of respondents is then recruited.
Today, for instance IPSOS is doing the Television Audience Measurement (TAM), then you have another research company ARA doing a radio survey etc. The problem is that each is using its own demographic benchmark; so when it comes to reading the figures and preparing a media plan, you are unable to compare TV data and Radio data because there is not a single sourcing of the data descriptors. Each institute is dealing with its own figures.
The importance of the achieved ES today is that we have one single reference and reading any data can now be comparable. You can compare radio audience to TV since we are using the same basic data. That's extremely important for media planning and to know where and how to optimize media investments.
Was it the first attempt to survey the establishments in the general population of Lebanon?
It’s the first time ever that an ES that is sponsored by the industry is conducted. Of course research companies may have their own ES. But that’s different in the sense it does not allow consistent data comparisons across companies.
Are you surveying establishments in the sense of businesses and organizations or households?
It is not a survey of establishments. It is a survey of households; it is a survey that establishes a common base for upcoming audience measurements surveys. They call it ES in the sense of benchmark survey and it actually should have been called Benchmark Survey. It would have been more accurate. It is used as a benchmark to evaluate if further surveys are representative of the population or if they deviate from an accurate representation.
It consists of a very large sample of people that you cover and you consider that this sample of people represents a small image of the population at large. So whenever you have a survey to carry out, you go to the people that have already been selected by this ES or else endeavor to match their profile
How did you draw your sampling in the absence of a census in Lebanon?
Yes Lebanon's population last census was done ages ago. But there was a census in 2004 that was carried out by CAS (Central Administration of Statistics). It was a census of buildings, households and establishments. It is a quite comprehensive census and provides a solid base for further population surveys that give good estimates of the Lebanese population. (The surveys that were carried out by CAS in 2006 and 2009 provided too quite reasonable estimates of the total Lebanese population and its profiles).
It is really a pity that we are now in 2019 and no other surveys of the same scale have been conducted recently.
To draw the sample, we based ourselves on the previously mentioned studies and used scientifically recognized methods and ran several projections to get an estimate of the population and its profile in 2019.
Then we compared these projections to other figures available from other sources to check which one was converging with these figures. We relied mainly on the census conducted by CRDP (Council for Research and Development of Pedagogy) yearly among the students population.
Though it is only limited to students it was possible to compare the students population from the CRDP and those obtained from the figures projected form the CAS surveys, and adopted those projections where the CRDP figures and CAS projected figures converged.
What was the sample you worked on?
6000 households.
Is it enough and representative?
Yes it's a quite large sample. The largest panel in the field is the TV panel. It is of about 600 households so we took the largest sample in the field we multiplied it by 10. This is a general rule of thumb practiced almost everywhere in the world.
Let's talk about the survey mechanics now. How a household is determined for eligibility?
The skip pattern that we adopt is the following. Let's say we pick a region, like Ashrafieh, we divide the buildings into clusters, and pick clusters randomly then we select one cluster out of every five clusters according to a skip pattern until we exhaust the whole sample of the area.
Within each cluster we pick houses using a random walk, using random numbers and skip patterns to select a building and a household. We arrive at a house and knock at the door. If nobody answers we don't replace the house, but we give it three recall attempts. After three times if no one opens the door, we consider it's an empty house and replace it but not within the same cluster.
If the door opens, we don't pick the head of the family for survey nor the first person that opens the door. We pick an ‘informed person’ to survey according to a selection called 'next birthday method': from among the pool of 'informed people' residing in the household we ask which person has its birthday the closest to the date of the survey. Of course they must be Lebanese nationals, and over 15 years old. This selection is important otherwise we will be doing our survey with people who barely leave their homes. And this is biased.
What about remote areas?
In rural area, we pick every three houses distanced from each other.
Is face-to-face the only method adopted? No survey over the phone?
It can be done but we did not. Face-to-face situation is considered to be the golden standard for ES although it is costly; yet the AA decided to pay the price to work with the best method available.
Running a survey over the phone is way cheaper and but is open to more inaccuracies. If you are facing a surveyor, it would be very difficult to declare an age that is way younger than what you are. The same applies to other variables.
How was the response rate in general? Do people easily open their doors and are comfortable to being surveyed by strangers?
We did face some difficulties in the city contrary to rural areas where people were more welcoming. Yet in general, Lebanese people are quite welcoming. In the US, or Canada for instance, they don't welcome you at home for such a thing. Survey happens outdoors in malls. It's called Mall Intercept. In France as well, it has become more difficult to do a door-to-door survey. They intercept people in the street, and ask them to come by to a place they would have rented to conduct the survey.
In Russia for instance, in every building there is a hall and the interviewer waits for the interviewee in the hall to survey them.
Every culture and place has its own means.
Does it affect the end result?
It does in a way but at the end of the day, there is nothing that is totally perfect and there are scientific procedures you need to apply to adjust for any bias.. We have to understand there is always an approximation in any human methodology and the purpose of any methodology is to tighten the approximation.
To raise the response rate, what we usually do is pick surveyors from the same areas where we are doing our survey, as they are less likely to be refused entry and contact in the household.
Before the start of the fieldwork, are interviewers trained for such a job?
This was a very particular and massive survey that demanded a team comprised of about 300 interviewers. We needed also to check the adequacy of the questionnaire. So the first step was to run 50 pilot questionnaires with 12 trained interviewers during four days. Then questionnaire was fine tuned based on learnings derived from the pilots.
The second step was to hold different training sessions at Reach Mass premises to ensure all interviewers and supervisors are able to handle properly the questionnaire; this included role-playing and mock interviews.
What can you tell us about the questionnaire? What does it entail and what idea can someone get about the surveyed household?
We ask about education level, profession, household characteristics, belongings - home/digital equipment, durables at home, distribution of TV sets by room, kind of TV sets, usage, means to watch etc. Same goes for every media: radio, magazines and newspapers, digital exposure etc.
It is also believed that the design and quality of the questionnaire have an impact on the respondent depending on how they are put together. How true is that and have you given attention to it as well?
We use the 'Memory Reconstruction methodology'. It is a well-known methodology for such surveys. For example, we don't ask "at what time did you put on your TV yesterday?" We ask "what time did you wake up yesterday; after you woke up when was the first time you put on the TV, or radio?" So we start by establishing with the person a benchmark, and asking him/her what he/she did next. Memory recall is optimized when it works in cascade from a point of origin. The person gives us a benchmark and starts thinking what he/she did next; read a newspaper, turned on the TV etc. Then we ask what was the channel that he/she picked first, from what time till what time. This kind of questions helps the person reconstruct the whole day he/she has passed by.
How long did this whole survey take you to finalize?
The fieldwork started in December 2018 and it's only till April 2019 that we were able to cover the whole territory.
What are the challenges that you faced all through? For instance how did you deal with fraud?
With regard to fraud, we had 3% and we dealt with it through three ways:
First we have a supervisor for each area, who goes back to visit again a household pretexting that an info is missing so he double checks that this household has really been surveyed and then he asks again a handful of questions to see if answers correspond to the ones previously given.
Then we do the same through phone calls, and re-ask some key questions.
The procedure is called back checks. 25% of the questionnaires were back checked for each interviewer. The World Bank standard puts the ceiling at 20%. We have exceeded that ceiling.
Further to the back checks , we have a software, that can detect some patterns in the answers, and indicates if the questionnaire or the response is valid or not. Invalid questionnaires are rejected by the software.
What would you say was the biggest challenge you had to face all through this massive operation?
First of all the biggest challenge was to do the projection of the Lebanese demographics, especially in the absence of updated population statistics. I described it earlier in an easy way but it took us a lot of iterations to figure out how to synthetise it.
In the field we didn't face major issues. Lebanese people are usually quite welcoming as I said. But there are some tactics we used to handle problems encountered that have to do with encouraging people to respond.
Once all this complex data is collected, how do you analyze it?
There is software for this of course. But we shouldn't forget here that Robert Ruud, a consultant to the AA, who has a long experience with media measurement, in northern Europe, and many developed and underdeveloped countries, was present all through the survey and we highly benefited from his know-how and international experience. We were working closely with him, as he supervised on the design of the questionnaire, which we discussed in-depth with him; also he intervened in the projections, the tabulations. There are methods to calculate certain things and we needed his advice to select the optimal methods. He provided vital recommendations.
Ruud's background in audit and media surely proved invaluable. He has an experience that spans over 40 countries where he worked in. He's sensitive to the cultural differences of each country. His intervention was priceless specially when we had to work on the definition of social class.
What in your humble view made Reach Mass win the bid over seven companies?
It is really difficult to know as many factors may be at play. But I think the fact that we have worked on media auditing on behalf of the industry since 2004 gave us some advantage .
From 2004 till now, we have been involved in over 20 audits in KSA, UAE, Egypt. Many of them commissioned by industry bodies and for a substantial number of them in cooperation with international auditors like CESP or Reply. This possibly gave us an edge, because we were able to look at the survey not only from a research agency point of view, but also from the industry point of view. Our proposal revolved around what the AA needed to do to align everybody around a single currency, ensure the survey will be adoptable by the industry, besides also presenting a solid and trustworthy methodology.
Are you satisfied by what has been achieved so far? And what's next?
This survey is the property of the Advertising Association and it was made to the highest quality, as a gold standard. Nowhere in the region such a survey has be achieved. The AA are pioneers in this field and I hope they will remain as such.
This survey will be updated again soon in October. Then starting 2020, we will have a monthly update.
Today, the ES will replace all previous descriptions of media target groups and sizes. All suppliers of media currencies must use the ES.
It is a very rich source of material that is now in the hands of the AA. The industry needed a trusted description of the Lebanese population and now they have it. It took some time to reach there but this is one big achievement. Now there is a single reference to all the researches that will be conducted from now on and there is no more calling into question the neutrality, accuracy, independence of any media research. And this is extremely important for the advertisers as the more they trust where they are putting their money, the more the industry will grow.
