Recently we have a little bit more free time on the project, so we decided to dedicate it to improving our skills in writing unit tests. And after some time, we came up with the situation, that our tests were very unstable. It is not easy to understand why your test fails when it fails every time when you run Karma. Even more complicated thing is to understand why your tests are failing from time to time, and from time to time they all passed, even if you did not change anything in your code.
I did not have a prepared solution for such type of the situation with unit tests, but I had big desire to make our unit test more stable. Here are the steps that I have done.
Checked the log. The complexity of my situation was that the error was not in our custom code. The log showed that the mistake came from the core jQuery code. The error text said something like the “undefined is not an object (evaluating c.nodeType)”. And there also was the source of the error and the line number on which error happened.
Tried to google error by the text and the source of the mistake (did not work in my case).
Changed jQuery. I was very confused by the name “c.nodeType”, as well as I was confused by the number of the line in which error happened. As we were using minified jQuery, the line number in the log was simply 2, and this information does not provide me any information at all. So, I decided locally change jQuery to not minified version. It worked, and I obtained the proper line number on which error happened. Also, I obtained not such frustrated name of the object on which error happened (elem.nodeType).
I tried to analyze the function on which error happened. As far as I understood, this functions was general for such common properties for jQuery node object as ‘width’, ‘height’, ‘outerHeight’ etc. The problem does not come from this place, this was caused by the function that invoked this one. At this point, I thought, that it would be great to find out the call stack for this function, for better understand from where this error came. In the nodeJS we do not have such simple instrument for debugging as in the browser (chrome dev tools I had missed you today in my environment). After some googling I have found such simple code that helped me:
console.log("caller is " + arguments.callee.caller.toString());
After running test one more time, I obtained on log function that invoked mine with elem.nodeType mistakes. By the simple search, I found this function and replicate this idea with finding up caller. I repeat this until I reach in this improvised call stack functions from our custom code.
What I found out that error was caused by trying to use classical jQuery methods (like ‘.css()’, ‘.height()’ etc.) in the method where we initialized the js component (in our working project we are not using the frameworks, but have component architecture). I assumed from the error text, that for some reason at that moment when we are trying to use this jQuery method in evaluation part of the unit testing, we do not have this jQuery node object at all. If it is not easy to debug tests, it is even more hard to debug them when the error happened on the evaluation part of the process. Why? I haven’t found any way in which I can check what exactly I have in this variable at the moment when the error happened on the evaluation stage. Maybe I had missed something, but still do not have an idea how I can check this.
In the cases when I do not truly understand what happening I try to think like a system in which this happened. So, here I was with the Karma, Mocha.js and Phantom.js. As far as I understood for some reasons in some cases my tests environment just do not have enough time to create the proper node on which this method can be invoked. So, firstly I was trying to make some magic with karma configuration as well as with general karma js, that we used to pre-set some variables. But, finally, the solution was a little bit different. In some cases, we have created jQuery node object directly in the describe section, and then invoked component on it in the describe or beforeEach section. When I had moved creation of the jQuery node object to beforeEach section above the component initialization the test started working more stable, and I stop receiving this one type of error (“undefined is not an object (evaluating c.nodeType)”).
What is the moral of all this story? Even if you do not have a lot experience in working with this system (I still feel a little newbie with unit tests), there are still possibilities to fix it by using engineering approach and common sense.