Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: 3D Renderer and Preview Export of Thumbnails #87

Merged
merged 43 commits into from
Aug 27, 2024
Merged
Changes from 1 commit
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
9cc66fe
init rendering work from @net-cscience-raphael (see Cineast)
rahelarnold98 Jul 8, 2024
9fa71ca
rendering of 3d models with embedded textures completed
rahelarnold98 Jul 10, 2024
25a279c
3D rendering and extracting of a preview finally works🥳, clean up fol…
rahelarnold98 Jul 11, 2024
f362c99
deleted duplicated and no longer used files
rahelarnold98 Jul 11, 2024
9dfa0a3
more cleanup
rahelarnold98 Jul 11, 2024
8a21952
deleted unused files
rahelarnold98 Jul 11, 2024
09a75c6
minor cleanup changes
rahelarnold98 Jul 12, 2024
ccac2de
3d model preview can now be a gif of different views
rahelarnold98 Jul 12, 2024
8678d62
removed print statements used while debuggung
rahelarnold98 Jul 12, 2024
b2d3e55
rendering window is hidden
rahelarnold98 Jul 12, 2024
2b15d7d
renamed model to model3d to be more specific
rahelarnold98 Jul 29, 2024
237c538
renamed ModelContent to Model3DContent to be more specific
rahelarnold98 Jul 29, 2024
f50b53d
removed deprecated and unused functions from core.model.mesh
rahelarnold98 Jul 29, 2024
a1ee346
changed OptimizerOptions to class
rahelarnold98 Jul 29, 2024
8714eca
removed empty class
rahelarnold98 Jul 29, 2024
335b167
Adds facility for external rendering.
ppanopticon Jul 29, 2024
170f676
Adds facility for external rendering.
ppanopticon Jul 29, 2024
641453f
Fixes combination of four images in ModelPreviewExporter.
ppanopticon Jul 29, 2024
9427706
Removes superfluous Renderer interface.
ppanopticon Jul 29, 2024
1fecf4f
ExternalRenderer now re-creates the external process on demand.
Aug 5, 2024
d9e67f8
rewrite of Java code to Kotlin
rahelarnold98 Aug 7, 2024
39f4ac7
changed GLMaterial to data class
rahelarnold98 Aug 7, 2024
fad75ca
changed Texture to data class
rahelarnold98 Aug 7, 2024
cc627ca
changed RenderOptions to data class
rahelarnold98 Aug 7, 2024
f6fd74c
changed RenderActions to enum
rahelarnold98 Aug 7, 2024
7e8d974
Camera is now a data class
rahelarnold98 Aug 7, 2024
70f3a36
removed joml dependency in core module
rahelarnold98 Aug 16, 2024
2a28b39
merged dev in feature branch
rahelarnold98 Aug 22, 2024
7f276a2
Minor optimisation as proposed by @lucaro.
ppanopticon Aug 23, 2024
8aecbef
Attempt at better error reporting.
ppanopticon Aug 23, 2024
16affa6
The -XstartOnFirstThread is only used on MacOS.
ppanopticon Aug 23, 2024
ad07a8f
Improves error reporting.
ppanopticon Aug 23, 2024
10042ba
GitHub action script now creates a screen (in an attempt to get GLFW …
ppanopticon Aug 23, 2024
b02197e
Removes logging.
ppanopticon Aug 23, 2024
6c47ac2
Removes change, since it doesn't seem to work.
ppanopticon Aug 23, 2024
50e609d
Dependency clean-up.
ppanopticon Aug 23, 2024
cba45f5
Adds display to ubuntu container and runs tests with display.
ppanopticon Aug 23, 2024
6961a83
Adds test reporting to CI/CD.
ppanopticon Aug 23, 2024
7a14cee
Merge branch 'dev' into feature/3d-preview
ppanopticon Aug 23, 2024
1d719a6
DISPLAY=:1 is now integrated into ExternalRenderer invocation of proc…
ppanopticon Aug 23, 2024
eaa3821
Fix.
ppanopticon Aug 23, 2024
02dfef1
Merged
Aug 26, 2024
1247b9c
Downgrades Protobuf dependency, since it seems to be incompatible wit…
Aug 26, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import org.vitrivr.engine.model3d.data.render.lwjgl.render.RenderOptions
import org.vitrivr.engine.model3d.data.render.lwjgl.window.WindowOptions
import java.awt.image.BufferedImage
import java.io.Closeable
import java.io.IOException
import java.io.ObjectInputStream
import java.io.ObjectOutputStream

Expand All @@ -17,27 +18,11 @@ import java.io.ObjectOutputStream
*/
class ExternalRenderer : Closeable {

private val process: Process
private var process: RenderProcess? = null

/** The [ObjectOutputStream] used by the [ExternalRenderer]. */
private val oos: ObjectOutputStream

/** The [ObjectInputStream] used by the [ExternalRenderer]. */
private val ois: ObjectInputStream

init {
val javaHome = System.getProperty("java.home")
val javaBin = "$javaHome/bin/java"
val classpath = System.getProperty("java.class.path")
val className = "org.vitrivr.engine.model3d.renderer.RendererKt"

val processBuilder = ProcessBuilder(javaBin, "-cp", classpath, "-XstartOnFirstThread", className)
this.process = processBuilder.start()

/* Initialize streams. */
this.oos = ObjectOutputStream(this.process.outputStream)
this.ois = ObjectInputStream(this.process.inputStream)
}
@Volatile
/** Flag indicating whether the [ExternalRenderer] is closed. */
private var closed: Boolean = false

/**
* Renders the given [Model3d] using the [ExternalRenderer].
Expand All @@ -47,21 +32,101 @@ class ExternalRenderer : Closeable {
* @param windowOptions The [WindowOptions] to use for rendering.
* @param renderOptions The [RenderOptions] to use for rendering.
*/
@Synchronized
fun render(model: Model3d, cameraPositions: List<Vector3f>, windowOptions: WindowOptions, renderOptions: RenderOptions): List<BufferedImage> {
/* Create request and write it to stream. */
check(!this.closed) { "ExternalRenderer is closed and cannot be used for processing." }
var process = this.process
if (process == null || !process.isAlive()) {
process?.close()
process = RenderProcess()
this.process = process
}

/* Send request. */
val request = RenderRequest(model, cameraPositions, windowOptions, renderOptions)
this.oos.writeObject(request)
this.oos.flush()
val response = process.send(request)

/* Read request. */
val image = this.ois.readObject() as? RenderResponse ?: throw IllegalStateException("Could not parse model.")
return image.images()
/* Return images . */
return response.images()
}

/**
* Closes the [ExternalRenderer] and the associated process.
*/
override fun close() {
this.process.destroy()
if (!this.closed) {
this.closed = true
this.process?.close()
this.process = null
}
}

/**
* A [RenderProcess] is a helper class that wraps the [Process] used by the [ExternalRenderer].
*/
private inner class RenderProcess: Closeable {

/** The [Process] used by the [ExternalRenderer]. */
val process: Process

/** The [ObjectOutputStream] used by the [ExternalRenderer]. */
val oos: ObjectOutputStream

/** The [ObjectInputStream] used by the [ExternalRenderer]. */
val ois: ObjectInputStream

init {
val javaHome = System.getProperty("java.home")
val javaBin = "$javaHome/bin/java"
ppanopticon marked this conversation as resolved.
Show resolved Hide resolved
val classpath = System.getProperty("java.class.path")
val className = "org.vitrivr.engine.model3d.renderer.RendererKt"

val processBuilder = ProcessBuilder(javaBin, "-cp", classpath, "-XstartOnFirstThread", className)
this.process = processBuilder.start()

/* Initialize streams. */
this.oos = ObjectOutputStream(this.process.outputStream)
this.ois = ObjectInputStream(this.process.inputStream)
}

/**
* Sends a [RenderRequest] to the external renderer.
*/
fun send(request: RenderRequest): RenderResponse {
/* Write request to output stream. */
try {
this.oos.writeObject(request)
this.oos.flush()
} catch (e: IOException) {
this.oos.reset()
throw IllegalStateException("Could not send request due to IO exception.", e)
}

/* Read response and return image. */
val image = try {
this.ois.readObject() as? RenderResponse ?: throw IllegalStateException("Could not parse model.")
} catch (e: IOException) {
this.ois.reset()
throw IllegalStateException("Could not parse model due to IO exception.", e)
}
return image
}

/**
* Checks if the process is still alive.
*/
fun isAlive(): Boolean {
return this.process.isAlive
}

/**
* Checks if the process is still alive.
*/
override fun close() = try {
this.oos.close()
this.ois.close()
} finally {
this.process.destroy()
}
}
}