{"id":241975,"date":"2020-04-30T07:55:15","date_gmt":"2020-04-30T07:55:15","guid":{"rendered":"https:\/\/imarticus.org\/?p=241975"},"modified":"2021-09-20T05:06:21","modified_gmt":"2021-09-20T05:06:21","slug":"what-are-some-tips-and-tricks-for-training-deep-neural-networks","status":"publish","type":"post","link":"https:\/\/imarticus.org\/blog\/what-are-some-tips-and-tricks-for-training-deep-neural-networks\/","title":{"rendered":"What Are Some Tips And Tricks For Training Deep Neural Networks?"},"content":{"rendered":"<p><span style=\"font-weight: 400;\">Deep Neural Networks aid AI applications such as image and voice recognition to function at unprecedented accuracy. A Deep Neural network is basically an array of several layers, where each layer sieves raw data into a structured mathematical model.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">The process of making the data flow through the various layers is called Deep Neural Network Training. In humans, we also start recognizing an object once we have seen it several times. If you saw just one \u201ccar\u201d in your entire life, you might not be able to recognize a car again if you saw a different model this time.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">In Data Science, this is easier said than done. Therefore, we have some tips and tricks that you can use when you sit down to teach your DNN to distinguish cars from trucks.<\/span><\/p>\n<p><b>Normalization is Effective<\/b><\/p>\n<p><span style=\"font-weight: 400;\">Normalization layers help group logical data points into a higher consolidated structure. An apparent increase in performance has been recorded when using Normalization.<\/span><\/p>\n<p><strong>You can use it three ways;<\/strong><\/p>\n<ul>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Instance Normalization &#8211; If you\u2019re training the DNN with small batch sizes.\u00a0<\/span><\/li>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Batch Normalization &#8211; If you have a large batch size, supposedly more than 10, a batch normalization layer helps.\u00a0<\/span><\/li>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Group Normalization &#8211; Independent of batch size, it divides the computation into groups to increase accuracy.\u00a0<\/span><\/li>\n<\/ul>\n<p><b>Zero Centering\u00a0<\/b><\/p>\n<p><span style=\"font-weight: 400;\">Zero Centering is considered as an important process for preparing your data for training. Just like normalization, it helps in providing accurate results later.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">In order to zero center your data, you should move the mean of the data to 0. You can do this by subtracting the non-zeroed mean of the data from all the data inputs. This way, the origin of the data set on a scalar plane will lie on 0, making it Zero Centered.<\/span><\/p>\n<p><b>Choose the Training Model Wisely\u00a0<\/b><\/p>\n<p><span style=\"font-weight: 400;\">One thing that you\u2019ll come across when you <strong><a href=\"https:\/\/imarticus.org\/certification-in-artificial-intelligence-and-machine-learning-by-e-ict-iit-guwahati\/\">learn Deep Learning<\/a><\/strong>, is that the choice of model can have a significant impact on training.\u00a0\u00a0\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Commonly, there are pre-trained models and there are models you train from scratch. Finalizing the right one that corresponds to your needs is crucial.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Today, most DNN developers are using pre-trained models for their projects as they are resourceful in terms of the time and effort required to train a model. It\u2019s also called Transfer Learning. VGG net and ResNet are common examples. <\/span><b>\u00a0<\/b><\/p>\n<p><span style=\"font-weight: 400;\">The key here is the concurrency of your project with the pre-trained model. In case you can\u2019t get a satisfactory model design, you can train a model from scratch too.\u00a0<\/span><\/p>\n<p><b>Deal with Overfitting<br \/>\n<\/b><span style=\"font-weight: 400;\">Overfitting is one of the most popular problems in DNN training. It occurs when the live run of the training model yields exceptionally good results but the same wasn\u2019t observed during the test runs.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">The problem is basically caused when the DNN starts accepting the attenuations as the perfect fit. This can be dealt with, using the technique of Regularization, which adjusts the problem of overfitting using an objective function.\u00a0<\/span><\/p>\n<p><b>Conclusion<\/b><\/p>\n<p><span style=\"font-weight: 400;\">Wish you\u2019d know more? Take up a <strong><a href=\"https:\/\/imarticus.org\/certification-in-artificial-intelligence-and-machine-learning-by-e-ict-iit-guwahati\/\">deep neural network training course on Imarticus<\/a><\/strong> and start your progress today. DNNs are becoming increasingly popular in data science-related careers. Just like everything else, you can use the first-mover advantage with pro-active learning.\u00a0<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Deep Neural Networks aid AI applications such as image and voice recognition to function at unprecedented accuracy. A Deep Neural network is basically an array of several layers, where each layer sieves raw data into a structured mathematical model.\u00a0 The process of making the data flow through the various layers is called Deep Neural Network [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":241976,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_mo_disable_npp":"","_lmt_disableupdate":"no","_lmt_disable":"","footnotes":""},"categories":[23],"tags":[680,805,1233,1733],"class_list":["post-241975","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-analytics","tag-artificial-intelligence-course","tag-machine-learning-training","tag-learn-deep-learning","tag-deep-neural-network-training-course-on-imarticus"],"acf":[],"aioseo_notices":[],"modified_by":"Imarticus Learning","_links":{"self":[{"href":"https:\/\/imarticus.org\/blog\/wp-json\/wp\/v2\/posts\/241975","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/imarticus.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/imarticus.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/imarticus.org\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/imarticus.org\/blog\/wp-json\/wp\/v2\/comments?post=241975"}],"version-history":[{"count":0,"href":"https:\/\/imarticus.org\/blog\/wp-json\/wp\/v2\/posts\/241975\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/imarticus.org\/blog\/wp-json\/wp\/v2\/media\/241976"}],"wp:attachment":[{"href":"https:\/\/imarticus.org\/blog\/wp-json\/wp\/v2\/media?parent=241975"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/imarticus.org\/blog\/wp-json\/wp\/v2\/categories?post=241975"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/imarticus.org\/blog\/wp-json\/wp\/v2\/tags?post=241975"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}