-
Notifications
You must be signed in to change notification settings - Fork 2
/
Copy pathpost_27_29.html
213 lines (190 loc) · 9.75 KB
/
post_27_29.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<meta name="description" content="">
<meta name="author" content="">
<title>Avinash Sen - Master Research Project Blog</title>
<!-- Bootstrap core CSS -->
<link href="vendor/bootstrap/css/bootstrap.min.css" rel="stylesheet">
<!-- Custom fonts for this template -->
<link href="vendor/font-awesome/css/font-awesome.min.css" rel="stylesheet" type="text/css">
<link href='https://fonts.googleapis.com/css?family=Lora:400,700,400italic,700italic' rel='stylesheet' type='text/css'>
<link href='https://fonts.googleapis.com/css?family=Open+Sans:300italic,400italic,600italic,700italic,800italic,400,300,600,700,800' rel='stylesheet' type='text/css'>
<!-- Custom styles for this template -->
<link href="css/clean-blog.min.css" rel="stylesheet">
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.4/MathJax.js?config=TeX-MML-AM_CHTML" async>
</script>
</head>
<body>
<!-- Navigation -->
<nav class="navbar navbar-expand-lg navbar-light fixed-top" id="mainNav">
<div class="container">
<a class="navbar-brand" href="about.html">Avinash Sen</a>
<button class="navbar-toggler navbar-toggler-right" type="button" data-toggle="collapse" data-target="#navbarResponsive" aria-controls="navbarResponsive" aria-expanded="false" aria-label="Toggle navigation">
Menu
<i class="fa fa-bars"></i>
</button>
<div class="collapse navbar-collapse" id="navbarResponsive">
<ul class="navbar-nav ml-auto">
<li class="nav-item">
<a class="nav-link" href="index.html">Home</a>
</li>
<li class="nav-item">
<a class="nav-link" href="about.html">About</a>
</li>
<li class="nav-item">
<a class="nav-link" href="contact.html">Contact</a>
</li>
</ul>
</div>
</div>
</nav>
<!-- Page Header -->
<header class="masthead" style="background-image: url('img/week_46_48.png')">
<div class="overlay"></div>
<div class="container">
<div class="row">
<div class="col-lg-8 col-md-10 mx-auto">
<div class="post-heading">
<h1>Week 27-29 </h1>
<h2 class="subheading">Development of 3D model and Custom dataset. Training and testing the Dataset for Evaluation.</h2>
<span class="meta">Posted by
<a href="about.html">Avinash Sen</a>
on April 12, 2020</span>
</div>
</div>
</div>
</div>
</header>
<!-- Post Content -->
<article>
<div class="container">
<div class="row">
<div class="col-lg-8 col-md-10 mx-auto">
<font size="6"><b>Development and Evaluation of Synthetic Data</b></font>
<p>
I used a simple deep network architecture, trained entirely on simulated data, to infer the 2D image coordinates of projected 3D bounding boxes, followed by perspective-n-point (PnP) for the segmentation of camera data. Its called DOPE – Deep Object Pose Estimation System.
<p>Followed these Methodologies:</p>
<p>1.Created a 3D CAD models in solidworks in (.stl/.ply format).</p>
<p>2.Imported in the BLENDER to give color, material, unit scales etc to object and export as (.fbx) file.</p>
<p>3.Imported the blender file in UNREAL ENGINE (v4.22).</p>
<p>4.Created the Dataset by capturing using NDDS plugin.</p>
<p>5.Trained the custom created dataset in CNN by Deep learning.</p>
<p>6.Saved the weights of the training for further System Integration.</p>
</p>
<font size="5"><b>Creation of Custom objects using the Blender tool</b></font>
<p>
Make a 3D model in solidworks or CAD etc.,then import a 3D model in Blender (.stl .dae ...) and export it as a fbx file then import that fbx file into UE4, fbx file is work in UE4.
<p>For more, __Download__<a href="https://www.blender.org/">__[[Blender]]__</a></p>
</p>
<p><b>Blender Setup for Creating the 3D Model object:</b></p>
<p>
<center>
<iframe width="640" height="360"
src="https://www.youtube.com/embed/bK2NJmRyP6g?autoplay=1&loop=1&playlist=bK2NJmRyP6g">
</iframe>
</center>
</p>
<font size="5"><b>Creation of Custom dataset using Nvidia Deep learning Data Synthesizer (NDDS) software in Unreal Engine (UE4).</b></font>
<p>
STEP 1 - Download the NDDS Documentation, wget <a href="https://github.com/NVIDIA/Dataset_Synthesizer/blob/master/Documentation/NDDS.pdfhttps://github.com/NVIDIA/Dataset_Synthesizer/blob/master/Documentation/NDDS.pdf">(NDDS) Documentation</a>
</p>
<p>STEP 2 - Installing & RUN the NDDS :
Open the Unreal Editor with the Dataset_Synthesizer/Source/NDDS.uproject,
a default level called TestCapturer will load as indicated at the top left hand corner of the 3D view port.
This level has a sample scene with a basic simulation capture set up.
</p>
<p>
<center>
<iframe width="640" height="360"
src="https://www.youtube.com/embed/qrhS-UxqNlQ?autoplay=1&loop=1&playlist=qrhS-UxqNlQ">
</iframe>
</center>
</p>
<p>
STEP 3 - Train the Created Custom model: Training code is also provided - <a href="https://github.com/avinashsen707/AUBOi5-D435-ROS-DOPE/blob/master/dope/scripts/train.py"> train.py</a>
</p>
<font size="6"><b>Training and Testing for Evaluation</b></font>
<p>
This network was implemented using PyTorch v0.4. The VGG-19 feature extractions were taken from publicly available trained weights in torchvision open models.
The networks were trained for 60 epochs with a batchsize of 16 for this Demo object. Adam was used as the optimizer with learning rate set at 0.0001.
The system was trained on an Dell Precision 7820 workstation (containing NVIDIA P4000 16GB GPU), and testing used same.
</p>
<p>For more, Download the Dataset from <a href="https://research.nvidia.com/publication/2018-06_Falling-Things">https://research.nvidia.com/publication/2018-06_Falling-Things</a> and
train the objects by <a href="https://github.com/avinashsen707/AUBOi5-D435-ROS-DOPE/blob/master/dope/scripts/train.py">https://github.com/avinashsen707/AUBOi5-D435-ROS-DOPE/blob/master/dope/scripts/train.py</a> to get respective weights.
</p>
<center>
<p><img class="img-fluid" src="img/training.jpg" alt=""></p>
</center>
<p><b>Debugging Object Detection on Devset : Results :</b></p>
<p>
<center>
<iframe width="640" height="360"
src="https://www.youtube.com/embed/G2MJ-pnd1hY?autoplay=1&loop=1&playlist=G2MJ-pnd1hY">
</iframe>
</center>
</p>
<p><b>Debugging Pose Estimation on Test Devset : Results :</b></p>
<p>
<center>
<iframe width="640" height="360"
src="https://www.youtube.com/embed/9iNDSaG5IdE?autoplay=1&loop=1&playlist=9iNDSaG5IdE">
</iframe>
</center>
</p>
<!-- Pager -->
<div class="clearfix">
<a class="btn btn-primary float-left" href="post_25_26.html">←Previous Week </a>
<a class="btn btn-primary float-right" href="post_30.html">Next Week →</a>
</div>
</div>
</div>
</div>
</article>
<hr>
<!-- Footer -->
<footer>
<div class="container">
<div class="row">
<div class="col-lg-8 col-md-10 mx-auto">
<ul class="list-inline text-center">
<!-- <li class="list-inline-item">
<a href="#">
<span class="fa-stack fa-lg">
<i class="fa fa-circle fa-stack-2x"></i>
<i class="fa fa-twitter fa-stack-1x fa-inverse"></i>
</span>
</a>
</li> -->
<li class="list-inline-item">
<a href="https://www.facebook.com/avinashsen707">
<span class="fa-stack fa-lg">
<i class="fa fa-circle fa-stack-2x"></i>
<i class="fa fa-facebook fa-stack-1x fa-inverse"></i>
</span>
</a>
</li>
<li class="list-inline-item">
<a href="https://github.com/avinashsen707">
<span class="fa-stack fa-lg">
<i class="fa fa-circle fa-stack-2x"></i>
<i class="fa fa-github fa-stack-1x fa-inverse"></i>
</span>
</a>
</li>
</ul>
<p class="copyright text-muted">Copyright 2020 © avinashsen707.github.io</p>
</div>
</div>
</div>
</footer>
<!-- Bootstrap core JavaScript -->
<script src="vendor/jquery/jquery.min.js"></script>
<script src="vendor/bootstrap/js/bootstrap.bundle.min.js"></script>
<!-- Custom scripts for this template -->
<script src="js/clean-blog.min.js"></script>
</body>
</html>