With the rapid advancement in technologies nowadays, new data is constantly and frequently being generated, processed, and stored for various applications. To ease up managing these huge collections of data, distributed systems are utilized. Distributed systems have the ability to be easily scaled and managed from different locations across the globe. One of the drawbacks of large systems is the difficulty faced when attempting to secure them. In this paper, we investigate the use and effects of different Deep Learning (DL) techniques such as CNN, BiGRU+Attention, and LGBM. We made use of two distributed training frameworks for the training of models in a distributed manner. The performance of various techniques is analyzed based on different evaluation criteria using the latest IDS dataset. Experimental results show that the models presented here are robust and efficient enough to detect different kinds of attack types with almost 100% accuracy. However, we observed that when the attack type increase from 3 to 5 the model accuracy drops from 99.9% to 96.34% for LGBM. Performance trade-off between centralized and distributed training is analyzed and observed that the same accuracy can be achieved in DIDS, but with differences in training time. Inferencing time shows very insignificant differences. The results of LGBM show that the duration, port numbers, packet size and number, and TCP flags are the most influential features in DIDS. Furthermore, we have shown the applicability of CNN and BiGRU with attention mechanisms for DIDS, as well as the deployment of two frameworks for distributed machine learning. More than that, LGBM has shown the most superior performance among all models in all types of classification. It is anticipated that this research will act as a guide for further research in DIDS.