ICMR PGI Online Course
About Lesson

Part I : Dr. Nusrat Shafiq

Part I : Transcript


Hello, this is Dr.Nusrat here, I will be talking to you about systematic reviews. I will be dividing this talk in two parts. The part one could be rather generals speaking about why they are important, what they help us do what kind of decision, they help us in making and some initial part about to how carry it out to would be more technical where it was see how to process of carrying a systematic review.

So let me start my presentation. I am try to bring up some real life situations which you may be encountering in day to day practice one of the thinks it believe really settings have encountered where we have sought some kind of evidence presenting to you. It was we often face with neonates tending with multidrug resistance about micro bacterium bomanni infection so the very lethal kind of bacteria it is increasingly being funding in our settings there issue there was the child was not responding to even three days of the drugs which was affirmation of the drug which was to which the drug was susceptible. This drug was colestine. It was being given through the intravenous route. So , we start inventering whether its adequately reaching the cerebo spinal fluid (CSF), where it suppose to reach or not.Hence we haveto look up the ecidence for the same and that was the one of the

situations. The other situation that came to us was the six year old boy who had history of loss of consciousness following a road traffic accident was admitted in emergency. He was given amoxicillin for profile access of pulmonary infection. Now we have antimicrobial associationship programme running in our institute and during an important part of that programme is to enable judicial use of antimicrobial. So we want to emphasize that it may this practice is not evidence based. How could be come up with this kind of information. We have to resolves to some kind of evidence for this.

Then again we have similar situation of sepsis we did not do because whether the patient specially neonates, they are difficult to diagnose sepsis due to bacterial etiology was quite difficult and so must be case many of you. You often resorte to biomarker to enable us to sort out the situation. So we want to which biomarker would be best able to answer address of delay now. So that was more about clinical practice situation where you may need to look for evidence inform of systematic review. Then you need for the second thing that we wanted to see use the second situation might drug a say talk about systematic reviews being very resourceful would be in policy making decision, one such example it came to us was as a part of ICMR for evidence based medicine for children health was to decided upon feasibility of administration of gentamicin for sepsis at peripheral healthcare workers level in our country. Gentamicin has to be given in by IV route. So it has its own set of complication. So lot of thought and had go into whether it should be made available in children to healthcare workers for administration or not so one way that be haded hand was looking carrying out a systematic review. Then if none existed previously. There are another example where they can be used for making decision at policy level. for example if we wants to know usefulness of social media engaging with adolescence or may carry out a systematic review. Assimilate all the awailable evidence in terms of clinical trials etc. and correlate and then see what the reason was made on inform decision. Then they could be useful for setting a research priorities. A lot of institutes like national institute for health agency of health care research and quality. These make use of systematic reviews for routinely making decision of healthcare policy and ICMR has started funding the center for EBM for child health. This is center has been produce systematic review to guide polices regarding vaccination, community based intervention etc. so this another important role that they have to that systematic review can plate. One want to know I mean does it have any commercial use, yes, you can’t sell systematic review but what can possibly being use. One more example your Cochrane reviews which are even taken up by food industry by the medical pharmaceutical industry where in they provide an evidence to in the lay public for advertising their products so this evidence if it comes in the form of a well conducted systematic review would be very helpful in direct consumer kind of advertising activity which otherwise makes use of vary literature which comes from very grace studies or improperly conducted studies, better to have a properly conducted systematic review to address to make use of such advertisement. So we are not trying to propagate that has one of it used but yes some decision is being made on the basis of properly conducted systematic reviews and it is more credible decision so we have just seen how various kind ways systematic reviews can help .So to just to summarized so it helps making recommendation and formulation guidelines. It helps in general in designing healthcare benefits in deciding about what to cover for insurance etc. and it helps in making some public policy decisions and then in routine decision making also systematic reviews are very helpful and this is one place where we make use of systematic review most often.

So systematic review only think that we have for making decisions. We have other tools is here. We have randomized control study, we have cohort studies, case control studies, case series and case reports which can help us in formulating in our decisions. So what is it about systematic reviews. So important and so valuable. The very fact it is review the focus research question. The focus research question that tries to identify, appraise, select and synthesize all high quality research evidence relevant to that question makes it very credible tool for making such decision. So what is important is that it is an assimilation of all very good quality data such that available about a particular problem and since its done. It follows a very rigrous methodology for carrying where it is conducted and conclusions drawn from it. There is a lot of values attached to it. So what is this link between systematic review and EBM. So I would say it rather follows this kind of cycle where first you have to ask research question. You have already being apprise about  previous presentations when you have acquire the data such that available various sources also we could exist in acquisition of these data and then we have to appraise of this data critically evaluated abstract, the relevant information analyze it and apply it, so that makes a very important tool for practice evidence based medicine. The important attributes of a systematic review. I would say which in it lend all the credibility associated to it a transparency and the process are very transparent. You have to outline the process before follow them. Then the way it is done. Methodology is such that the chances of bias is if not obliterated it totally minimized and interest so that you can make the decision and if it in cooperate a process statistical correlation of data and its makes so these the important attributes for any kind of scientific decision processes so what is it how are these review different from narrative review usual kind of reviews where we topic is taken and all the literature is discussed about the topic. How these systematic reviews different from those narrative reviews and does this process of carrying out systematic review in cooperate these evidences, transparency , accuracy and reduction of biasness. we have clear specific objective which is stated upright in the beginning. Once you have the research question or the query in your hand, not necessarily a research question or you lay down the objective of the review very clearly. so process is very transparent what are the objective going to be? There is protocol written out how it is going to be done and very often protocol is published. There is certain sites you go registered your systematic review and can have protocol published. So it makes very transparent. Then the methodology uses. For example the literature you going to search for the data that are available. It was a very systematic process. You have to outline it a priory. So the purpose of that kind of search is that the risk of selective sampling of studies is reduced. We would like to emphasize more about this patient we gave often saying that reviews which are conducted by certain pharmaceutical companies may include only studies show clinically significant priority their product of drug. So by their selected selection they gave already move in favour of their own product on the other hand, which is properly conducted with make sure all the possible eligible study which are included in our review and not excluded. They have not any significant finding. So then you have very consistent evaluation available information such as outcomes in study quality is carried out. You have a very stringing very methodical way of accessing the quality of study. It is not arbitrary, it are certain parameters on which these studies are evaluated for the quality. There are also a very systematic method also exist for extracting the information out of these. So all the processes that we there are in a manner that biases are reduced. The reader is allowed to make decision ashe/she moves along reading review. So at each step he would know ok this review has been, for example this review which is carried out is not much so good quality of study. So they will know that this review has shown a significant benefit of a particular intervention. But the studies which were include in the review were perhaps that of meet good quality. So they are informed. Readers are inform when making a decision. Since the method for calculating and collating the data very precise and well defined. There is an element of accuracy there and then as more data emerges in subsequent years and months to came. These review has scope of being updated. So that’s those were major reasons are systematic reviews given more credibility or other kind of reviews. So what important think that may came to your mind is that where does this meta analysis come in all this thing we have to understand this point of meta-analysis is just a subset of syatematic reviews. It is the statistical component of systematic review where in mathematical calculation are under taken to collate data that is obtain from various studies. So not all systematic reviews would have meta-analysis.

So it is important to know that rigorously can conducted systematic review have been very good citation and this was illustrated by the particular study which was published in BMC medicine. So that’s are the more reasons to undertake them. Ok so backbone of systematic review is systematic search. So there is lot of data which are available on the electronic media and the we have it on the books in the original article written in unpublished data. So how much to take or how much to leave is an important question. So there is two things that we gave to keep in mind how sensitive we have to be regarding our process of data acquisition. One thing is there if it is abundance of data it may be very tricky situation where you may end up having a lot of data. Some of them which may nor data relevant. A large majority of data or on the other hand if you make your search process or strick end of just of few studies miss not. Very important set of data. There has to be set and balance and balance was depend upon a context in which you making researches. Quick review during the rounds. You could be looking for more specific answer rather then and you would just enter of two or more three systematic review of study that should be sufficient. By research process that you undertaking then you may be need to more through about you searches. Now there are various numerable numbers of sources we can get your systematic reviews. That once we referred to most commonly are pubmed and Cochrane. Cochrane is the one of the important once where systematic reviews are published regularly and update these also find a place to pubmed. Pubmed could be direct and indirect access.

Such strategy need to be defined making search will have details of the search strategy. In a separate lecture ou gave to use proper operator for getting the maximum of such maximum of things as well as such and you must take help of your librarian while you are doing so you certain steps that you should be following for taking a search. List the databases plan to search. Decide it was appropriate to limit search can temporarly. You want to limit four years, five years only. So that could be only important in certain situation then you may decide that you want to have only RCT, that you have observational studies and write down your research question. Identify the key concepts of your questions.

The next important role is thing of the alternate words that could be use for these concepts. For example you may miss of just write epilepsy if you write seizures in epilepsy inspite of epilepsy you may get other set of data. So you should be aware of this Boolean operator should be used judiciously . Then if need the filter should be used. If you don’t want any studies which are conducted through example in may not wanted so you see this out and they all systematic review should having component of hand searching. So that’s one think the best thing is always library.

One is important to do your searches problem properly. You may end of what is called the published we have referred to it earlier or negative studies, negative studies term left and this very commonly problem. So error based to give and idea about how much is the possibility of a publication in a particular systematic review. One of the important role is your inverted funnel plot what the plot does this the results of the studies and along with the outcomes and against the variation so you get in if it is symmetrical these distribution is symmetrical and inverted funnal plot kind of what you see is then all kind of studies such that small kind of studies large variation with the end may be large effect this is outcome very overrated outcome and also those with large sample sizes less variation and small outcomes of small effect sizes would all be distributed evenly. So that the true outcome possibility of publication bias. This is just give your idea how systematic review may have their search strategy outline. You can go through it.

So that is just about that inclusion and exclusion criteria is second think. What are you doing to include and what are you doing to exclude and what are kind of studies that you doing to include so that you find them priority means before you conducting a systematic reviews. So you may need to say about that I am going to studies conducted in say example. Pediatric population setting for example. Which are conducted in tertiary care centers which evaluate for example your ICD placement outcome variables you may say what is the duration of a person stay which should be part of the outcomes when are mentioned in the systematic review and at least have a follow up for 6 th month, for twelve month or which are of a certain level of methodological quality. So it not important to have all of them have inclusion criteria dependent upon when you find the problem that is the  operational defination of the condition is very important studies used different definition for a particular collection. Asthma a large variation of definition may exist in various. So you have to say what is the definition of asthma in epidemiological studies such you go to use for the systematic review that was reasoning example and then you have to describe your intervention properly. For example one of our systematic reviews, we had reviews cholesterol lowering diet has the following induction of intake of. So reduction in cholesterol intake, reduction etc. so you have to very precise about your intervention. And then you have to outline the outcomes properly or could be primary outcomes and could be secondary outcomes. So you have to say at the end of the table should be able to see for example percentage of coverage of target population mortality in the first form of life. These were the some outcomes that we used for out gentamicin review in the community setting then secondary outcomes less important comes to bring need to referral to high center treatment failure defined to change empirical and antibiotic. So it depend case to case but general principle followed is what is most important question that you want to answer form your systematic review would be your primary outcome. Rest of question became the secondary outcomes.

Lets talk about the systematic quality of studies we will first have to see what are the quality of studies that is use in yours systematic reviews. Different methods that have been defined there is methods of jadad there is another method that is followed by Cochrane handbook described particular method that you have followed in list out on the parameter of which quality was evaluated. It could be this is an example we are seeing here Cochrane handbook quality assessment methodology. We test studies possibility of selection bias for possibility of in allocation concealment again would take care of selection bias. Then performance bias in detection while studying is blinded on not lost to follow up was how much it was address or not and there was any selective reporting of the study. So only these parameters you will evaluate each and every study and come out with its conclusion which is rather this is an objective method for coming which id exact not so objective with this way of objective perception. So quality of studies would be very important why is quality of studies given so much of importance systematic reviews. It simply because it helps the being make inform about the degree of dependent reliance that can put on resist obtained from a particular systematic review. So this was about the first part of the systematic review and these wouldbe the some of the references have use for presentation.

Thank you.

Part II : Dr. Nusrat Shafiq

Part II : Transcript

Dr. Nusrat Shafiq (lecture II) So we have just finished part one of systematic review and now we have to know two of the systematic reviews. Now we give to know part two of the systematic review. Part 1 we have seen how we should derived our search strategy and then how we should about inclusion and exclusion studies then what kind of studies we should taken and how quality of studies should be assessed. So we now at the stage where we may have 1,2,3,4 or many studie it had and then how we processed from them. So the second part of the systematic review that we going to do. Now we going to addressed from the story from here on. So we will start from a straight point. we have the studies with us and we have assessed them as per they quality concerned as they good quality studies ok quality studies or poorly conducted studies and what was the possible lacuna in those studies. These have already being addressed. Thus what we do with paper that we have to be with us. That is obtained the data that we need to obtain for synthesis carrying out our systematic review. For addressing above particular queries. So how to be go about it. Even this method has to be a very systematic method and it is suggested that it should be carried out at least by two independent people. Those people are working independently and then these data should be collected and overseen by somebody else. A third person who would and see what there are any differences in the information obtained by two people who are working independently and if there are any search differences then there should be a risk and a relook needs to be data. For this usually a simple data extraction form should be generated a query where you list out different kind of data that you would obtain for example what would be some information about the study population the age of this population or the severity of the disease. It could be those information then what are the intervention use for example its particular drug in what dose of how many days was the drug given. So that kind of information would be written down. No. of participants in the interventional group or the control group. In each study used to be taken out very carefully and the no. of participants for each outcomes should be written down. Then what was the results that are obtained what are the outcomes for example if it was a study on cholesterol lowering diet therefore the effect on NDL in the intervention group and control group they have separately listed out and when we talk about a listing out of these reasons we have to write down the specific values for example if total cholesterol reduced from 200 to 180 mean value should be written as well as variation parameter like should also be listed very precisely. Many times of the see all this information not available. All would be presented in a different manner. For example instead of giving mean and standard deviation confidence interval would be given. So we can work these out. Sometime these cannot be work done. So we have to go write to the author and obtain the data if they gave that particular data. Sometime it may so happens that you selected these studies and to realize that the some of the data which you want not there you came have to write to the authors and try to get that data from them. If he give then well and good otherwise you put such a attempt and you could not obtain the data . we would generally end up two kind of data. It would be qualitative data on quantitative data. Qualitative data could be improved or not improved, But again there would be consider more appropriately as categorical data and then the corresponding figures would be in terms of proportions,percentages. So these have to be listed out exact numbers may be given say 10 out of 20 patient responded and 10 out 20 did not respond so this information has to be putted down in the data extraction form. You can have continuous data where it can be something like your height of the child, weight of the child. So these would measure the information that need to the noted down is the mean value, the median value and standard deviation, interquartile range , so whatever they final outcome parameter that you going to obtain from the study have to be defined a priory. for example if it a mean weight loss going to put down on the paper, this has to be known beforehand that it is a mean weight loss that have been going to this abstract. Sometimes these value are not available you to build them up.Build them up means out of Whatever type of data is available and obtained it from investigators. so as we have already mentioned that meta-analysis is a statistical component of systematic review where all such data obtained from different studies are collated by using a very stranging methodology. For example if it is about a cholesterol reduction study and say I have five studies in hand and five studies were reported various degrees of reduction in cholesterol levels. In will have all the information about these reductions exactly how much was the reduction. Now what I do. I use a methodology for collating these result of the five studies. It is not simple addition and dividing by the no. of studies it gives very curious studies values of where methods are defined. There are two types of methods are defined which are generally used one is the randomly effects method. If there is lot of variation in the result. Then this is the method that is used commonly and then other one is the fixed effects method. If variation is not the significant then fixed effects methods may be used. If It is a diagnostic test where in the things of and parameters of concern would be sensitivity and specificity or true positivity rate, true negativity rate. Then what we make use of use of these value to generate the protocol is summary operating curves . So before we start collating the data we have to be understand. Do we have to carry out a meta-analysis? We often with these question that’s carry about how to carry a meta-analysis with the data said that we have very often we end up giving a large search do not combine apples and oranges because if somebody is evaluating for example what you could say is a loss of daily wages and is expressing it is in Indian nationalrupees and there is another set of person who must evaluating for example study has reported the no. of workers are lost. So these are two different parameters you cannot pool them together. So they have to be reported as such without collating. If meta-analysis is permissible if it is possible. It should be carried out if methods of reporting the result across the study is similar. For example all of them has reported a mean reduction in total cholesterol or mean reduction in LDL level elevation in HDL level then could go ahead with collating the data with using any of the model that we have talk about random effects and fixed effects model. It depending upon the variation across studies you will chooses this models and then you used a software to undertake this collation. Commonly used for software is revman. There are many others which are available. Revman can be freely down load . All these calculations very well be carried manually but.it is very tedious process undertakes this if its categorical data like something happens and something not happen for example death occurs or not occurs. That kind of data then we to kind of statistics that we use to summarize the collated data. The odds ratio, relative risk, desired ratio and with 95% of confidence interval. That if it something continuous data like growth weight or weighted to months following intervention. Then it is the mean and standardised means with 95% confidence interval. That would be your summarize results after collation. One important thing that we have to be bear mind is called heterogeneity. When we have the data with us we must evaluate of heterogeneity and the degree of thing heterogeneity. Heterogeneity can arise of because various reasons it could simply differences intervention, differences in the patient population and it could be something statistical heterogeneity something which is simply becuse difference across the study due to any of the reasons but they acquire statistical relevance it makes a difference in theway that this is there. So there ways to every meta-analysis where in should care addressed how much of heterogeneity was present among these studies. There are specific tools for example we have chi square test and Q test. Which those software undertaking the meta-analysis and can get the resultswhich data they are carried out. Certain values which are Q values which could be chi square value. It help us decision making and about the presence of heterogeneity and the degree of heterogeneity that is presented. If the heterogeneity that is quite significant then what we have to keep in mind that we have to do use a random effect model for data synthesis. The other way of doing assessing heterogeneity could be meta regression. so what is meta-regression now we are talking about it. Look about meta-regression. It is nothing but evaluation of factors which could have resulted in difference in the result can be seen. So there are again itcan be carried out any of the software. Stata has the facility of this undertaking meta-regression but there are other software which are available for this thing. Let us deeper to this statistical heterogeneity the little variation it means the result of studies is about the expected that expected by chance it cause statistical heterogeneity. There are therefore of uncontrollable factors and I mention again heterogeneity into Cochrane. Q test a value which is of 0.10 uses the cut off. When the other test which is commonly used the chi square statistics. It expresses the result in term of percentages. It describe the percentages of variation of studies such that used due to significant heterogeneity rather than by random chance. You will chi square value 50%, 75% and so on. Then total messages is the heterogeneity is significant. You should using random effects model. Once we have the data how do we put in the software and we have command for editing the data. We end up what are the further forest plots or this is figurative representation of the results that have been obtained. These plots are very beautiful plots in this time in sense that they give you a reader a very good idea about outcomes of study in one glance. So you lets just pose at one of the forest plot that we have at hand. So most of your software would give result in forest plot once you put the data for collation here So if you see here we this forest plot has been taken from of our publication about it does we have fact. All the data of individual studies and then we told them we want to use it a fixed model therefore heterogeneity was not significant so it not only gives us results of the overall different kind of intervention the different kind of intervention that are used for the CETP inhibitors, Naive cell and fibraids. It lets out the individual studies that went into making of that into synthesizing the data. For example we just take CETP inhibits how may studies here we know there were we can see here 6 studies. We get an idea about what were the are the no. of events in each arms of all these studies put together if you concern there it is term weight here that weight indicates the how much of importance it should be given to study that importance of the weight is derived from the two factors sample sizes and larger the sample size of these study the more like it have a way to weight second thing is the variation in the reporting outcomes smaller the variation larger is the weight of its study. If we can see the weight of studies the range from 0.1% to 22.1%. so you can see its very clearly the largest studies had been assigned a great of weight. Now which is more on the right hand side and we see a term of odds ratio. Each individual study also an event occurring in the intervention group against that occurring control group are detected and then summary odds ratio is generated. So if you see the individual odds ratio is 1.58 – 1.37 and so on. And the summary event odds ratio is 1.12 and the same is represented on the right hand side in the diagrammatic manner. The figure in the bracket gives you an idea confidence interval of each these odds ratios that you obtained. So it is important to look at these confidence interval therefore it tell you how precise was the measurement observation for example you have a very wide confidence interval make you believe that there was consideration in the variation of result was obtained in the study. So one must look at the variation and the must look at the confidence interval the widest the narrowness of the confidence interval. And so all the things that we have been talking or summarize in this particular single plot suppose you could not to do a Meta -analysis. Studies were not ranges kind of studies that you could not correlate them so addressing how do summarize the results. So it not can’t you can do say something like coverage of target like two field include in study. The studies are mentioned there and it was found that out of 5919 neonates presented 5510 visited by voluntary health workers. It has coverage of 93%. So it gives an idea to read what are the kinds of coverage that you could on the studies. That you had in hand. You can describe the data like this. Of you have an individual patient data then you go individual patient meta-analysis. It is considered as very important. It increased the possibility to inform more complex statistical analysis. The power of analysis increased, subgroup of analysis undertaken they have a high validity. You can read defined your subgroup with or individual patient data have to it hand and opportunity to examine the consistency of subgroup effects across studies. However the biggest drawbacks is that we do not have very good data bases which are universally available for example for somebody sitting in India if where to do meta-analysis. It is very difficult to be obtain this data from any resource. These are increasingly being undertaking and other improvisation upon meta-analysis was trials sequence meta-analysis. I give you an example of trials sequence meta-analysis where they trials are getting added the results are evaluated sequentially and are presented. You get this idea about the tempo rant profile valuing evidence there. Then once we have on this at right hand we have correlated the data we have analysis our results. We have obtained the results that we have so how do we report. Certain reporting standard said be outline. For example the PRISMA for observational studies. These must be followed. The flow of study diagram is a very important diagram. Here in what we can see is. It tells you how many initial records did you get? How many could you include why that once was finally excluded and then how many full text articles you could take for extracting the data and then finally how many articles could be evaluated. So to summarizes your flow of number of studies that different stages of undertaking analytic process. So all again say it was be had the computers. They started in MS Dos format and then we move down to windows format and then move down to laptops and i-Pads. Since exactly how meta- analysis also evolved. You would think is very complicated no longer. If you do not have to do manually. It is quite easy. The important part is your data extraction. Have to be very correct and précising. Your data extraction processor. Once that is done rest is take care off. So that is all about systematic reviews was seen we have to go about the stage we started from patient where we want to know what was the importance and even progress to point a how to be learn start meta-analysis with a systematic review process and we understood what are the difference between systematic review and narrative review. We finally how do we correlate and tehn how do we derive conclusions from this data. So these will taken more in detail of subsequent lecture that is you see how to derive the conclusion. How to read them. How to interpret odds ratio and relative risk and related mean difference so there would taken is subsequent classes. So that is all about systematic review. We want to take queries.

0% Complete