Comparative Performance of Gradient Boosting and Random Forest for Smart Home Device Classification

The proliferation of IoT devices in smart environments has brought significant challenges related to device management, security, and communication integrity. Effective classification and recognition of these devices are crucial for enhancing management processes, strengthening security protocols, and ensuring seamless communication. This study compares Gradient Boosting (GB) and Random Forest (RF) classifiers using a dataset containing network traffic data from 1,900 smart home devices. The classifiers were evaluated based on their ability to categorize devices into ten distinct categories, each representing a specific type or function of IoT devices. Utilizing performance metrics such as accuracy, precision, recall, MCC (Matthews Correlation Coefficient), F1-score, and AUC (Area Under the Receiver Operating Characteristic Curve), both models demonstrated strong classification capabilities through 20-fold cross-validation. Results indicated that Gradient Boosting achieved a precision of 92%, recall of 90%, and an F1-score of 91%, slightly outperforming Random Forest, which recorded a precision of 89%, recall of 87%, and an F1-score of 88%. However, Random Forest demonstrated superior performance in AUC with a score of 0.94, excelling in distinguishing specific device types. Both models encountered challenges in classifying devices with overlapping features, underscoring the necessity for further feature refinement and tuning. The ROC curve analysis provided insights into the actual positive rate versus false positive rate trade-offs, while confusion matrices highlighted specific categories where misclassifications occurred. This study underscores the potential of integrating advanced machine learning techniques to improve IoT device classification and facilitate smarter, more secure environments.