Featured SRI PhD Projects
\ \ \
Deep Reinforcement Learning-based Industrial Robotic Manipulations
In this research project, the basic research question is how to develop smart, intelligent, and self-learning algorithms for industrial robotic manipulations such as Pick-and-Place, which can produce effective and enhanced results. The aim of this research study is to build an efficient self-learning framework for industrial robotic manipulations in order to maximize the production quality and quantity of the various industries.
Main Objectives
The main objectives of this project after extensive literature review can be listed as follows:
One
Development and dissemination of results of a reinforcement learning-based algorithm to learn tasks such as pick-and-place in a non-visual environment.
Two
Development and dissemination of results of deep reinforcement learning-based algorithm to perform and learn manipulation tasks in a vision-based environment.
Three
Development and dissemination of deep reinforcement learning-based algorithm to learn multiple different manipulations together in a vision-based environment.
The training and testing beds are designed in the V-REP simulator developed by Toshiba R&D. V-REP have physics engines such as Bullet and ODE providing complete real-time experiences. For motion planning, Open Motion Planning Library (OMPL) is deployed to gain real-time inverse kinematics calculations in order to achieve dynamic motion planning of robotic arms. In the non-visual approach, off-policy and on-policy temporal difference algorithms Q-learning and SARSA agents were designed to make our Jaco robotic arm (6 degree-of-freedom) to learn pick and place different shape objects at three different alignments (left, center, right) and speeds of the belt (slow, medium, fast) with the help of proximity sensors.
For the vision-based approach, deep Q-networks (DQN) were utilized to make the agent UR5 robotic arm (6 degree-of-freedom) learn the pick-and-place task of different regular and irregular 3D objects with the help of orthogonal and perspective vision sensors. Pixelwise parameterization technique was used in order to generate action-value maps. RGB-D heightmaps created from the RGB-D data transmitted from vision sensors were given as states to the designed DQN. Prehensile and Non-prehensile robotic manipulations were learned together by updating the testbed and increasing the number of networks in the DQN.
Enhancing Cyber Attack Prevention and Detection using Application Process Tracing
Nowadays, software enterprises are being targeted by more advanced cyber-security threat models. Consequently, more sophisticated means of protecting software organisations are in high demand. Also, microservices are trending for being amongst the most popular software application design architecture. The aim of this thesis is to explore how application process tracing can be applied to enhance cyber-attack prevention and detection.
OBJECTIVES
We propose two objectives for this research. The first objective is to observe how the prediction of future events in an application thread can help identify potential targets and thus enable cyber-security personnel to take proactive defensive measures. This approach is valid for general business application processes. The second objective is to investigate how anomaly detection approaches can be applied to microservice application process tracing and detect seeded cyber-attacks.
Approach 1.
One approach for addressing the first objective is to employ a machine learning model to learn general business application processes and functionality to provide a contextual oversight of the application’s infrastructure. This can be done by applying process mining to observe the execution paths of application processes. An alternative method is to employ a deep learning model to discover the contextual oversight of the application process. We trained a LSTM model to learn the sequential dependencies for existing processes and subsequently made predictions in ongoing process instances.
Approach 2.
For addressing our second objective, we considered microservice application process tracing. The functionality of a microservices application can be monitored and logged using distributed tracing. Anomaly detection is defined as the discovery of irregular instances or patterns within a data series. To enhance detection of cyber-attacks, frequency distribution-based anomaly detection was performed to identify anomalous trace activity within a synthetic data set of traces. This machine learning model was tested by simulating a brute force password guessing attack against the application.
Part 1 of Thesis:
A deep learning model trained to discover a contextual oversight of a general application process and process flow prediction was carried out by employing using this deep learning model to train with four different data sets with the aim of enhancing cyber-attack prevention.
Part 2 of Thesis:
An open-source microservices application, SocialNetwork is run and general traffic is generated and logged using distributed tracing. A brute force password attack cyber-attack is seeded and detected using frequency analysis-based anomaly detection.
Part 3 of Thesis:
The microservices application is runs again for this project, microservice traffic is generated, a graph convolutional neural network, the DCRNN model, is trained to discover the spatial and temporal dependencies of the data and threshold-based anomaly detection is performed detect seeded cyber-attacks based on anomalous microservices traffic.
