Taking the AWS Certified Machine Learning – Specialty Exam

Posted on by marin.stoytchev

After taking the AWS Certified Cloud Practitioner exam in April, following my plan on May 20th I took successfully the AWS Certified Machine Learning – Specialty exam. Here I would like to share my preparation for the exam and the experience with taking the exam itself.

Preparation for the exam

The first thing I would like to share is that this exam is a completely different beast compared to the AWS Cloud Practitioner exam. Yes, I knew that the Cloud Practitioner is the easiest of all AWS Certification exams, but I had no idea how different these two exams are. In my previous post, I wrote that I hoped in two-three weeks to be able to prepare for and to take the Machine Learning – Specialty exam. Now, I realize how naive that thinking was.

The material that you need to cover for the AWS Certified Machine Learning – Specialty exam is enormous. I would roughly separate the material into two different categories.

First category: General Machine Learning (ML) knowledge which includes, but is not limited to the following:

  • Popular ML algorithms like K-Means, KNN, PCA, Random Forest, SVM, XGBoost, etc; when it is appropriate to use certain model and when it is not.
  • Deep Learning (DL) networks like CNN, RNN (LSTM and GRU); their structure and usage; activation functions, etc.
  • Evaluating ML and DL model performance; when the model is underfitting and when it is overfitting; what the appropriate measures for improving the model performance in each case are.
  • Performance metrics like Precision, Recall, F1 score, ROC (AUC); what each one means and when it is appropriate to use it.
  • Model tuning – hyperparameters for different algorithms and best tuning practices.
  • Transfer learning – what it means and how it is put in practice.
  • Statistical distributions, outliers, data imputation and engineering, data visualization.

Second category: AWS related topics which include, but are not limited to the following:

  • AWS ML services – Kinesis, Athena, QuickSight, others.
  • AWS data storage, migration and transformation services – Glue, Data Pipelines, AWS Batch, Step Functions, …
  • AWS SageMaker algorithms – BlazingText, DeepAR, Factorization Machines, Image Classification, IP Insights, K-Means, KNN, Latent Dirichlet Allocation (LDA – not to be confused with Linear Discriminant Analysis), Linear Learner, Neural Topic Model, Object2Vec, Object Detection, PCA, Random Cut Forest, Semantic Segmentation, Seq2Seq, XGBoost.
  • AWS models development, deployment and security.
  • AWS higher level services – Amazon Comprehend, Amazon Translate, Amazon Transcribe, Amazon Polly, Rekognition, Amazon Forecast, Amazon Lex, Amazon Personalize, Amazon DeepLens, Amazon Neo, …

For the four general topics and their representation on the exam, please, see https://d1.awsstatic.com/training-and-certification/docs-ml/AWS-Certified-Machine-Learning-Specialty_Exam-Guide.pdf.

As one can see, there are a lot of topics/subjects to learn for this exam. I would say that because of my studying and working on Machine Learning algorithms and models over the past year, I knew to greater extent the subjects covered by the first category. However, on the subjects covered by the second category I knew very little. It was clear to me that preparing on my own would take a lot of time and the outcome would most likely be uncertain. From previous experience, I turned to the trusted and well-established approach of taking online courses for preparation for the exam.

Courses I took for the exam preparation:

  • Udemy “2020 AWS SageMaker, AI and Machine Learning Specialty Exam” by Chandra Lingam. This course helped me a lot with the general ML knowledge because it goes beyond the introductory Data Science and ML courses I had taken before. Also, it is heavily focused on Amazon SageMaker with a lot of hands-on exercises on developing and deploying SageMaker models. I would say that this course is geared not so much towards preparing people for the AWS ML Specialty exam, but more towards providing good knowledge and practice on ML in general and Amazon SageMaker in particular. The course has also a full-length practice exam at the end. However, in my opinion the questions there are more focused on the material covered in the course and do not cover some portions of the questions in the real exam. Regardless, I would recommend this course to anybody interested to expand their ML knowledge and learn about Amazon SageMaker with good hands-on exercises.
  • Udemy “AWS Certified Machine Learning Specialty 2020 – Hands On!” by Frank Kane and Stephane Maarek. This course is specifically geared towards preparation for the AWS ML Specialty exam. Despite the title including “hands on”, there are actually no hand-on sections. This course resembles more a PowerPoint presentation discussing each exam topic in great detail, but lacking in in-depth explanations. It helps with preparation for the exam, but in terms of delivering practical knowledge it cannot be compared to the course mentioned above. Also, it does not include a practice test, which in my opinion is unfair to the students who take this course since practice tests are a big part of the preparation for the exam.
  • Additional courses with full-length practice tests. I took two such courses on Udemy. The first was “AWS Certified Machine Learning Specialty: Full Practice Exam” by Frank Kane. This practice exam should have been included in Frank’s course to begin with. It covers well most of the topics with many questions being very similar to the questions on the real AWS exam. The second course was “AWS Certified Machine Learning – Specialty Practice Exams” by Jayaram Narayanan. It has two practice tests – one full-length with 65 questions and the second is shortened to 25 questions only. These practice exams are similar to the practice exam provided by Frank Kane. However, I would say that there were some specific questions not covered by Frank which were nearly identical to the questions I encountered on the real exam. I would like to mention that there are other practice exams one can find on the web, but I would say that the above practice exams provide good practice for preparing for the exam at the moment. In my opinion, going over other practice exams or trying to get as many practice exams as possible does not add much value.

A word of caution: AWS is adding new services and updating the existing ones over time. So, while all of the above courses are relevant at the moment, I expect that six months from now new questions to the exam will be added and one should look for updated courses for preparation.

I spent four full weeks (including weekends) studying the material from the two courses and going over the practice exams. I took a first try on the practice exams at the two-week mark. At that time, I barely passed one of the practice exams (the Chandra Lingam one) with a 78 % score and failed the other two exams with a score of about 70 % vs. the passing score of 75 %. So, I continued studying for two more weeks until I was passing multiple times all of the practice exams with at least 96 % score (translates to two incorrect answers out of 65 total). At some point, I realized that I have memorized the correct answers for all questions. That’s when I started taking the practice exam every third day instead of every day. After four weeks of studying I decided that I am ready to register for the real exam.

Registering for the AWS Certified Machine – Specialty Exam

Although this might not be a big deal for some people, I would like to mention it to help others save time, keep them on track, and in the right state of mind. When I took in April the AWS Certified Cloud Practitioner exam, I went online to register right after deciding that I am ready to take the exam. I had no issues scheduling the exam for the next day and successfully took it. For the Machine Learning – Specialty exam I decided to follow the same path. On May 6th, I decided that I am ready to take the exam and proceeded to scheduling it. Because of the COVID-19 epidemic Amazon has announced in April that all AWS certification exams can be taken online. When I selected the online exam option, I was unpleasantly surprised to find out that the first available date is May 20th. This was a great disappointment for me. I have studied non-stop for four weeks, taking repeatedly the practice exams for the last few days and was ready to go. And now I had to wait two weeks to actually take the exam. I was angry with myself that I did not think ahead. With Amazon allowing all their certification exams to be taken online it was natural to expect that a lot of other people will do the same as what I was doing and the demand would be overwhelming. So, at least in the current situation as related to COVID-19, if somebody is planning to take any such exam online, my advice would be that they check in advance for available dates and plan accordingly. This would save them time and disappointment.

After my initial disappointment, I decided to take a couple of days break. I was thinking that after the break I might work for a week on some ML project and then shortly before the exam get back to the practice tests to get ready. However, when I tried to work on something else, I constantly found myself thinking about different exam topics and what I could learn more to be better prepared for the exam. I realized that this exam meant too much to me and I won’t be able to focus on something else. So, after two days of unsuccessfully trying to work on a non-related project I got back to studying for the exam. This time, besides reviewing the material from the two courses and going over the practice exams, I spent significant amount of time reading Amazon materials and focusing mostly on Amazon SageMaker (see https://docs.aws.amazon.com/sagemaker/latest/dg/whatis.html).

Taking the AWS Certified Machine Learning – Specialty exam and my impressions from it

Since I already had experience from taking the AWS Certified Cloud Practitioner exam online, this time the check-in process went absolutely smoothly (see my check-in experience with my first AWS exam in the previous post). About 15 minutes after starting the process I was allowed to start taking the exam.

The exam has 65 questions in total and the time allowed is 180 minutes (three hours). The questions are a) multiple choice type with one correct answer and three incorrect responses (distractors) and b) multiple response type with two or more correct responses out of five or more options.

I am not going to give a detail breakdown on the questions here. It is just not possible to remember all of the questions on the exam. And as I said earlier, the practice exams I took provide a good representation of the type of questions on the exam. Instead I would like to share few observations that stuck with me.

  • Questions difficulty level: Here, I am comparing to the AWS Cloud practitioner exam since this is my only reference point. The questions for the Machine Learning – Specialty exam are much more difficult. There are very few easy questions. There are no questions where the presence of key words in one answer could point you to the correct answer. In almost all answers (at least two answers as a minimum) the key words are there, so one cannot focus on key words alone, but needs to decide if the services, tools, methods or the combination of these is valid.
  • Questions type: In terms of (a) and (b) type questions, my impression is that there were only few questions of type (b). Most of the questions were of type (a) – i.e. single correct answer from four possible choices. In that regard the practice exams I took had significantly more questions of type (b) – multiple response. This might be a result of the random draw of questions for a person taking the exam (I am not sure if this is how it works), but from my experience be prepared for mostly type (a) questions.
  • Questions detail: Many of the questions describe a business problem in great detail and ask for the appropriate action. That makes the questions long and complex which requires careful reading and understanding. This in turn, takes time. This is where taking practice exams over and over really helps. With practice one becomes accustomed to this type of questions and the lengthy descriptions, and it becomes easier to grasp the essence of the question. Thus, one does not need to spend too much time reading through the long (and some times convoluted) question and trying to understand what they are being asked. So, practice, practice, practice with the practice exams. It truly pays off.
  • Questions you cannot get the correct answer by reasoning through the problem: There are questions that one cannot really apply any reasoning to arrive to the correct answer. You either know which is the correct answer or you don’t. On this type of questions, best approach is to go with your most educated guess and move on. There just will be questions that you will not get the right answer and you have to accept this. That brings me to the next point.
  • My strategy for taking the exam (and accounting for time, as well): Because I have already accepted that it won’t be possible to answer all questions correctly, I decided to devise a way to gauge the likelihood of passing the exam at the time of taking it. For me, it boiled down to counting the number of correctly answered questions or rather possibly incorrect answers which is easier since this number should be (hopefully) smaller. The 75 % passing score means that one has to answer 49 questions correctly. Thus, in the case of possibly incorrect answers the absolute threshold for me was 15 such questions. In order to have some margin of error, I set the threshold of possibly incorrect answers to ten. Using the lesson from my taking the AWS Certified Cloud Practitioner exam last month, when I was answering the questions I went with my best answer without spending too much time on any question in particular and not flagging any difficult questions for later review. This allowed me to answer all questions in about 60 minutes with 120 minutes left. Thus, I had plenty of time to review carefully all of the questions again. Going second time over each question, the most important thing for me was whether I believed that the answer is correct or not. If I had any doubt, I would count it as “not clear”. I did not use the word “wrong” because labeling an answer as wrong could bring a lot of pressure when the number of “wrong” answers start going up. This is more of a psychological effect which might not be applicable to other people, but it works for me. “Not clear” still allows for some possibility of the answer being correct; “wrong” does not. After reviewing carefully all questions and answers, I counted nine “not clear” answers. During the review process, I changed the answer of one question only. Changing the answer did not put it in the category of correct answer. The answer was still in the “not clear” category, but I believed that the revised answer is more likely to be correct than the answer I gave initially. I finished the review process in 40 minutes and since I only counted nine possibly incorrect answers I went on and finished the exam.

One thing that deserves strong attention is that sitting in front of your desk and staring at your laptop screen for nearly two or in the worst-case scenario three hours without getting up (not allowed in the online exams) is surprisingly hard and exhausting. So, one needs to be in good shape and be prepared to go through a marathon.

I finished the exam and after answering a brief survey was informed that I have successfully passed the exam.

Now, I have my AWS Machine Learning – Specialty Certificate as shown below.

Moving forward, I do not think that after getting this certificate I have the knowledge and experience that comes from many years of working in Data Science and Machine Learning. However, I learned a lot over these six weeks of preparation. Now I know much more than before I started preparing for the exam. And in the fields of Data Science and Machine Learning, constantly learning new things is part of the job and I have proven that I can do that. I am planning to apply my new knowledge by working on several new projects on my own. Possibly, I will revisit some of my early projects which I have published on GitHub and use new evaluation metrics in some of them. At the same time, I will be looking for the right opportunities to apply my knowledge in a working environment.

I do hope that sharing my experience with the AWS Certified Machine Learning – Specialty exam will help some of you out there. Good luck to all who have decided to follow a similar path!

Related Post