Modeling protection devices is essential for performing accurate stability studies. Modeling all the protection devices in a bulk power system is an intractable task due to the limitations of current stability software, and the difficulty in updating the setting data for thousands of protection devices. One of the critical protection schemes that is not adequately modeled in stability studies is distance relaying. Therefore, this dissertation proposes two different methods for identifying the critical distance relays for any contingency, which are required to be modeled in stability studies. The first method is an iterative analytical algorithm and the second method is an ML-based method. The performances of both the methods are evaluated on the Western Electricity Coordinating Council (WECC) system and the results show that to have an accurate assessment of system behavior, modeling the critical distance suffices, and modeling all the distance relays in not necessary. Furthermore, modeling various generator protective relays in stability studies is also crucial. However, no comprehensive framework has been developed that provides guidelines on proper representation of generator protective relays in stability studies and evaluate their impact on the dynamic response of a system. To fill this gap, this dissertation proposes a comprehensive systematic framework which enables proper representation of generator protective relays in stability studies, thereby increasing the accuracy of these studies. The framework is tested on a particular area of the WECC system and the behaviors of different generator protective relays is evaluated.Finally, this dissertation proposes a comprehensive machine-learning (ML)-based online dynamic security assessment (DSA) method that broaden the concept of online DSA by predicting loss of synchronism (LOS) in generators, and the operation of critical protective relays in a power system. The performance of the method is tested on the WECC system in the presence of different noise levels and missing phasor measurement unit (PMU) data. The results reveal that the method can provide precise and fast predictions and is robust to noise and missing PMU data. Therefore, the method can be reliably used in power systems to enhance situational awareness by providing early warnings of impending problems in the system.