Note: This blog post represents my opinion and cannot be considered a scientific document. However, it can be considered a discussion subject. Since it is my opinion and it has a high potential not to be true. If you disagree after reading it, just forget about it. I used this blog to write my thoughts about some topics. This is my current opinion; I may change it over time if I learn new things. But feel free to contact me if you want to discuss it.
It is well-known among other computer science communities that the software engineering community is outstanding in creating new names, terms, and acronyms to be used in publication and presentation. This is seen as a “Reinventing the wheel” concept. I think that is true in many communities, but I also agree that the software engineering community is overdoing it. Search-based Software Engineering (SBSE) is an example of this case. When I started my Ph.D. study, I searched for a new catchy and sexy topic. I was accepted as part of the software engineering program. So, I knew my field, but not my topic. During my starting research, I felt that many ideas are just rephrasing old ideas but with new catchy names. One of the newly published papers during that time was on Search-based Software Engineering (SBSE). The name had all the characteristics of “Reinventing the wheel.” Simply, it is on using optimization methods in the software engineering fields. But from my search in the software engineering field, I immediately knew that this is possibly another new catchy name that will stand for a while, and perhaps it will be good to finish my Ph.D. on this topic. I picked some new optimization methods and used them in software testing, part of the software engineering field. I optimized the test cases by searching for effective test cases using some optimization methods. I got new and nice results and published results in high-quality journals and conferences, and Walla .. I got my Ph.D.
During that time, I studied in-depth optimization algorithms in terms of type, parameters, algorithms, and other aspects. Since the time of my Ph.D. was limited and the focus was not that broad, I kept this for a while until I finished my study. When I finished my Ph.D., I started exploring optimization methods. But I also kept in mind this SBSE. The community had grown, and many people had followed the trend by that time. But, I always asked myself, the optimization and search algorithm were there, and software engineering was already there, so why did we create this term? The optimization and search algorithms were used a long time ago to solve software engineering problems. So, what was new?
What made the SBSE more trendy was the “Reinventing the wheel” habit that happened to the optimization community. During that time, several well-established and mature optimization algorithms in the research community have been used for a while, like Genetic algorithm, Simulated Annealing, Particle Swarm, Tabu Search, and several other algorithms. But these were not enough for that community to publish new papers. The trend of creating new nature-inspired optimization algorithms started. So, every day, there was a newly published optimization algorithm in a journal. At the time, it was very catchy. You can imagine that now, we have an optimization algorithm for each animal name. This trend also made the SBSE even more viral. So, I started using some of these newly invented optimization algorithms in my work. I got some new results. However, I found that they are not always good. That is due to the randomness of these algorithms. Then I used another algorithm, and I got new results. Then I used two more and got new results. But, I did not find an algorithm that gives good results all the time. I also did not find any algorithm that could be the winner for all cases—Mmmm.. something strange here.
So, the trend in the SBSE was following the trend of the optimization community. For example, new algorithm X is published in the optimization community, then immediately after that, a paper appeared in the SBSE community, using X in solving Y problem. Y here is a software engineering problem. And, of course, you should add some SBSE explanation in the paper. And, here it is, you have a good paper to be published. But then, this became a recipe for many papers. You just need to change X or Y in that sentence, and it becomes a new paper. I even followed this trend like any other community member. However, I went back to the original idea and found that by a good parameter tuning of the conventional and old optimization algorithms, you can even get the same results that you get with the new optimization algorithms. I came back to the optimization algorithms and found that the algorithms look different. Still, in fact, they are the same and even the changes presented are not leading to any new conceptual change in the strategy in general. Of course, what I’m saying is just my observation, but there are also some scientific studies and results showing that recently. Going back to those newly developed optimization papers, it looks like they used the same experimental setting and the same implementation, and most of them were published by the same set of authors. So, to be realistic, if you have implemented one of them in a code and used the benchmarks, you may be able to develop another new algorithm in one day or even less than one day. What you need is just a good imagination to pick an animal and be able to express the behavior of that animal in mathematical expression, for example, when the animal is searching for food. Then, use those optimization expressions in the old code and write a new paper. The authors of these papers got thousands of citations because of this habit and the SBSE community. I guess nowadays, that community got to my point that this is not useful and in its maturity level. We don’t need any new optimization algorithms. Still, we need to improve the classical algorithms to be adaptive to the new kind of applications, like making them self-adaptive or distributed.
Coming back to the Search-based Software Engineering, there are some other issues apart from the naming. One of the problems I found in the papers published within the SBSE area is that most of the authors of these papers have a very shallow understanding of the optimization field. I have read many papers in very good journals but with many conceptual mistakes in the optimization algorithms. Part of this problem is also because the reviewers, who are also from the SBSE community, are more interested in the software engineering problem than the implementation of the optimization. A few years ago, I visited some well-known conferences with the SBSE theme, and I found that many of the attendants and presenters don’t have that good knowledge of optimization, including some well-cited people in the field. But, for the SBSE community, it is the same problem as the optimization community but with a different style. When the optimization reaches some saturation state, the SBSE community invested in finding new problems to be solved by the optimization algorithms. In my opinion, this is better than what happened in the optimization community.
The people in the SBSE community did not stop at this point, and it seems they liked that trend creation. They invented a lot of new sub-names, like search-based software testing, Search-based energy consumption, learning-based software engineering, and search-based xxxxx. However, not everybody can make a trend, just like social media trends. It depends on the author’s name, how many followers he/she has, how many conferences he visits per year, and how he/she does the marketing.
In my humble opinion, there are some problems behind this trend creation and “Reinventing the wheel” habit. First, there is a high demand and greediness for publication as a metric of job evaluation in academia. Writing a paper with these trends and showing that you are following a new topic has a higher chance of being accepted, especially when you cite some of those newly published papers to show that you are up to date and following the trend. Second, lack of topics, in which it is easier to follow the trend rather than having some new ideas. Third, the power and influence of some members of those communities. They are hired by good universities that can pay to visit many conferences and then use those conferences to make connections with other people for marketing.
Bestoun S. Ahmed
Contact: bestoun@kau.se
When I started my master study early 2007, immediately I got a fellowship from my university. A condition in the fellowship contract was to publish at least two journal papers, without mentioning which kind of journal. I was entirely new in this business, and I had nobody to guide me. When I got the first set of results from my research, I started writing a paper. At that time, I got an email from a journal inviting me to publish a paper. The publication price was so low, almost 50$. They wrote in the “About” page of the journal that the impact factor of the journal is 1.2. Immediately, I finish the paper and sent it to them. Surprisingly, after almost one day, they sent me an email, Congratulations, your paper has been accepted! Without any review !!
I was so happy, to have my first paper getting published. There was nobody to tell me that you have to check the impact factor and the quality of the journal from WoS. Anyway, after few years, I realized what the problem is. In fact, I regret about those papers published in those journals. The quality and the content of the papers was not bad.
After these years, I started writing for impact factor or high-quality journals. However, still those papers are like a nightmare in my academic life, and they came back to me from time to time. The sad thing in academia is, people are looking at you negative points. I have to say that those papers were negative points in my academic career but that is just an early stage trial for publication. I have been rejected in many job application partially because of them. I got negative feedback like “The applicant has published in fake journals” One of my grant application also rejected recently because the reviewer took a bad impression on me and he said, “this investigator has many papers published in the fake journal.” Well, as I said they are not many, they are just four papers.
In fact, there are different kind of fake journals and quality journals nowadays. Fake journals are well-known now. If they are not listed in WoS, then my advice is to do not publish because it does not count for you. There is one good trick that I like to do sometimes. There are many good journals listed in Elsevier or WoS, and they did not get impact factor yet. Most of the time, those journals are published papers in open access style. Here, you will get two benefits. First, you will get better citation since they are open access, and second, they will get impact factor soon since they are open access and following peer review process.
So, don’t make the mistake that I did in my academic life. I’m trying to erase them as much as I can by taking care of my papers and my work. I believe that putting more efforts in your job, taking care about what you are doing, update your self, be honest and love what you do, then finally people will appreciate and recognize you.
When I started my Ph.D. research around 2009, my supervisor suggested working on t-way testing. The whole software testing field was new to me, although I came with a good background in benchmarking of distributed systems. But now I can say it was not really software testing rather it was benchmarking. I started by reading several books about the software engineering and then other books about software testing. Later, I understand the whole process, and I was able to implement my first draft of code to generate t-way test suites. I met a friend of mine who worked on the use of search algorithms in control engineering. I realized that could be useful in my work and after several weeks of discussion I started using particle swarm optimization (PSO) for t-way testing. I was the first who implement the PSO in t-way. However, I was not the first who use search algorithms. One other researcher used Simulated Annealing (SA). After graduation and getting my degree, I realized that, instead of t-way tasing, I also have something to say about these search algorithms. I have used the t-way testing as a case study then and I with my supervisor implemented several improvements of the search algorithms and applied those improvements into t-way testing.
We enjoyed doing this job actually, and we implemented many new variants of those algorithms by making some small changes and contributions to those search algorithms. We published several papers in that direction. In fact, in software testing in general and t-way specifically, few famous names think that they are guard and god of this field. That is not wrong actually. However, they started criticizing us, indirectly by giving tough comments on our papers and also rejecting many of our papers. In fact, there was one reason behind this behavior. They thought that we are repeating ourselves and even they were asking for new bounds in t-way testing. In fact, those reviewers missing our aim in the papers although we tried to elaborate that in the text.
We aimed to say something in the search algorithm part not the t-way testing part. The t-way testing here was just a case study. Now you may ask why we did not use the standard mathematical benchmarks in optimization methods? Well, we thought to do that also, but as far as we have a real problem why we use some theoretical problem? Using this approach, already many people cited our papers by using our improvements on the search algorithms.
In fact, these days we stopped writing those papers although we still have many things to say. But, when the editors see the t-way testing phrase, they have few choices for reviewers, and when they get our paper, they just reject it without reading.
I think the generation research of t-way testing is getting into saturation nowadays. It is very hard to publish a paper in that direction especially when the paper goes to those well-known names in this field. However, there are many things that we can say in the application of t-way testing. T-way testing is not applied very well to the real-life problems and also in the industry. So, the contribution now is in the application, not the generation.
Over the past seven years, I have reviewed more than 150 scientific papers in my field. You may ask why I’m reviewing all these papers? First, I’m getting the most up-to-date papers by this work. Imagine that people are reading papers when it is getting published, but I’m reading them even before publication. Second, you know what others do in research, hence, the review broadens your scope. Third, I see this review process as a public service. I tried my best to help authors to improve their works by giving useful feedback, and I hope others look as critical and constructive to my own work.
Recently, I have tried to record some of them on publons.com . Check out my account on publons, here. I have also served as an Editorial board member for some journals, like Applied Soft Computing, by Elsevier (I.F. 3.5). I got many certificates, appreciations, and awards for my reviews and feedbacks for those papers I reviewed. Over these years, my experience in the review has been improved gradually. I reached to an understanding that, reviewing, criticizing, giving feedback to others’ works is an art. Of course, you may know this already, but I reach it by experience. Now, I can confirm that my first review in 2011 is not like my review a few days ago. Over the time, you become more stable and able to give wise advice. In fact, while I’m trying my best in this process, sometimes, as an academic, I’m suffering from “bad” review on my papers. Of course, by “bad” review I don’t mean that they reject or ask for a revision, but because the reviewer (sometimes the associated editors also) does not know to criticize or give feedback in principle. Now, I know most of the journals are providing guides for reviewers, among other guides; however, these guides are too formal, and they don’t contain real advice. Combining all these reasons, I decided to write down those points that I recognized during this time and put them all together to form some informal guidelines from my experience. I will categorize them into three categories, (1) Accepting a paper to review, (2) Reviewing a paper, and (3) Submitting a review. I will update these guidelines from time to time when I recognize some new issues.
Accepting a paper to review
This stage is when you get an invitation to review and you have to accept or decline it.
Reviewing a paper
This stage is the actual review stage when you accepted the invitation to review.
Submitting a review
This stage is when you finished the review and now you are about to submit it.
Note: Taking this content without mentioning the name of the author is not allowed.
Copyright © 2024 Bestoun S. Ahmed, Ph.D. - All Rights Reserved.
We use cookies to analyze website traffic and optimize your website experience. By accepting our use of cookies, your data will be aggregated with all other user data.