1.0-rc2

- change default host to VPS
- allow overriding of default cache target
- increase read timeout
- fix doubly stripping extension

Building.md

@@ -0,0 +1,31 @@
+## Building
+
+Prerequisites:
+- maven 3
+- openjdk >= 8
+
+Build: 
+```bash
+./build.sh
+```
+
+Executables:
+```bash
+dxvk-cache-client
+dxvk-cache-server
+```
+
+Archlinux:
+
+See [PKGBUILD](arch/PKGBUILD)
+
+## Implementation problems
+
+### Identifying a game
+
+Possible Solutions:
+
+- Just the exe's filename. After a bit of discussion the only possible choice: https://github.com/doitsujin/dxvk/issues/677
+- ~~SHA1 of the exe.~~ Don't want to loose the cache if the application is updated. Games built using an engine can have the same exact binary.
+- ~~Steam game id.~~ The most robust and my preferred solution, but would make it exclusive to Steam.
+- ~~Exe name plus parent directory.~~ ~~Still suboptimal but right now what I opted for. Assumes users don't go around changing the installation folder name. Should work well for Steam.~~

FAQ.md

@@ -0,0 +1,13 @@
+## FAQ
+
+#### Why is setting an environment variable necessary?
+
+The client will create a symlink inside each wine prefix it encounters when scanning from `drive_c/dxvk-cache-pool` to `$XDG_CACHE_HOME/dxvk-cache-pool`.
+
+All caches will be written to $XDG_CACHE_HOME/dxvk-cache-pool, so if your wine prefix is missing that symlink or the DXVK_STATE_CACHE_PATH isn't set DXVK won't find the cache.
+
+#### What if I don't want to set the environment variable globally?
+
+If you don't want to set it up globally you have to set it before running wine, otherwise it won't use the shared caches.
+
+You can probably configure it in Lutris for the wine prefix you want to use or create a wrapper script.

README.md

@@ -8,37 +8,11 @@ Client:
 - Fetches missing DxvkStateCacheEntry's and patches the .dxvk-cache.
 - Submits DxvkStateCacheEntry's generated locally since the last run.
 
-Server (users do not need this):
-- Centralized storage. Provides REST interface to access caches and signatures.
-
-
 Not affiliated with the DXVK project, please don't blame him if this destroys your cache files.
 
-## Building
-
-Prerequisites:
-- maven 3
-- openjdk >= 8
-
-Build: 
-```bash
-./build.sh
-```
-
-Executables:
-```bash
-dxvk-cache-client
-dxvk-cache-server
-```
-
-Archlinux:
-
-See [PKGBUILD](arch/PKGBUILD)
-
-
 ## Usage
 
-Both client and server require Java >= 8.
+The client requires Java >= 8.
 
 ### Client
 
@@ -60,40 +34,19 @@ usage: dvxk-cache-client  directory... [--download-verified] [-h] [--host
     --verbose                  Verbose output
 ```
 
-#### Environment
+### Environment
 
 For wine to use the shared caches you should set the DXVK_STATE_CACHE_PATH environment variable and point it to either:
 - `$XDG_CACHE_HOME/dxvk-cache-pool` will work for most people
 - or `c:/dxvk-cache-pool` if you did sandbox your wine prefix (`winetricks sandbox`) as in that case wine can't access your home directory. You need to run `dxvk-cache-client` against all your wine prefixes in this case.
 
-It doesn't affect steam/proton, proton is doing it's own thing and overrides that variable.
-
-
-##### The convenient way
-
-Set it up globally.
-
-See [dxvk-cache-pool.sh](dxvk-cache-pool.sh) for an example you can put directly into `/etc/profile.d/`. The arch package already includes it.
-
-
-##### The hard way
+This environment variable can be setup globally by putting [dxvk-cache-pool.sh](dxvk-cache-pool.sh) into `/etc/profile.d/`. The arch package already includes it.
 
-If you don't want to set it up globally you have to set it before running wine, otherwise it won't use the shared caches.
+It doesn't affect steam/proton as proton is doing its own thing and overrides that variable.
 
-You can probably configure it in Lutris for the wine prefix you want to use or create a wrapper script.
+### Example
 
-
-##### Why this is necessary
-
-The client will create a symlink inside each wine prefix it encounters when scanning from `drive_c/dxvk-cache-pool` to `$XDG_CACHE_HOME/dxvk-cache-pool`.
-
-All caches will be written to $XDG_CACHE_HOME/dxvk-cache-pool, 
-so if your wine prefix is missing that symlink or the DXVK_STATE_CACHE_PATH isn't set DXVK won't find the cache.
-
-
-#### Example
-
-Assuming you store your wine prefixes in `/usr/local/games/wine`, you can run it like:
+Assuming the game files are stored somewhere in `/usr/local/games/wine`, you can run it like so:
 
 ```bash
 $ ./dxvk-cache-client /usr/local/games/wine
@@ -111,21 +64,27 @@ updating 2 caches
 found 0 candidates for upload
 ```
 
-You can pass multiple directories. The directories should contain wine prefixes.
-It will search for exe files in the passed directories and automatically update the .dxvk-cache's for you.
+You can pass multiple game directories. For each passed directory, the client will recursively search for exe files and automatically create the .dxvk-cache's for you if they did not already exist.
 
+Alternatively, if the .dxvk-cache's already exist (i.e. you have already ran each game at least once), you can pass no directories at all like so:
 
-### Server
 ```bash
-$ ./dxvk-cache-server -h
-usage: dvxk-cache-server [-h] [--port <port>] [--storage <path>]
-       [--versions <version>]
- -h,--help                 show this help
-    --port <port>          Server port
-    --storage <path>       Storage path
-    --versions <version>   DXVK state cache versions to accept
+$ dxvk-cache-client 
+target directory is: /home/owner/.cache/dxvk-cache-pool
+scanning directories
+ -> scanned 3 files
+preparing wine prefixes
+looking up remote caches for 2 possible games
+ -> found 2 matching caches
+writing 0 new caches
+updating 2 caches
+ -> QuakeChampions: is up to date (2008 entries)
+ -> UE4-Win64-Shipping: is up to date (2928 entries)
+found 0 candidates for upload
 ```
 
+If you are using a sandboxed wine prefix, then the directories should be wine prefixes instead of game files (with game files and binaries stored in the prefix).
+
 ## Security
 
 All state cache entries are signed.
@@ -148,18 +107,6 @@ You can opt to only download cache entries which are signed by verified users (`
 
 See [Verification](Verification.md).
 
+## Miscellaneous
 
-## Implementation problems
-
-### Identifying a game
-
-Possible Solutions:
-
-- Just the exe's filename. After a bit of discussion the only possible choice: https://github.com/doitsujin/dxvk/issues/677
-- ~~SHA1 of the exe.~~ Don't want to loose the cache if the application is updated. Games built using an engine can have the same exact binary.
-- ~~Steam game id.~~ The most robust and my preferred solution, but would make it exclusive to Steam.
-- ~~Exe name plus parent directory.~~ ~~Still suboptimal but right now what I opted for. Assumes users don't go around changing the installation folder name. Should work well for Steam.~~
-
-
-
-
+For frequently asked questions, see [FAQ.md](FAQ.md). For building, see the [Building.md](Building.md). For server documentation, see the [server README](dxvk-cache-pool-server/README.md).

dxvk-cache-pool-client/src/main/java/com/ignorelist/kassandra/dxvk/cache/pool/client/CachePoolClient.java

@@ -52,7 +52,6 @@
 import java.util.NoSuchElementException;
 import java.util.Objects;
 import java.util.Set;
-import java.util.concurrent.ExecutionException;
 import java.util.zip.GZIPInputStream;
 import org.apache.commons.cli.CommandLine;
 import org.apache.commons.cli.CommandLineParser;
@@ -124,6 +123,9 @@ public static void main(String[] args) throws IOException {
 			if (commandLine.hasOption("min-signatures")) {
 				c.setMinimumSignatures(Integer.parseInt(commandLine.getOptionValue("min-signatures")));
 			}
+			if (commandLine.hasOption("cache-target-dir")) {
+				c.setCacheTargetPath(Paths.get(commandLine.getOptionValue("cache-target-dir")));
+			}
 
 			final ImmutableSet<Path> paths=commandLine.getArgList().stream()
 					.map(Paths::get)
@@ -213,8 +215,9 @@ public synchronized void downloadVerifiedKeyData() throws IOException {
 					try (OutputStream out=Files.newOutputStream(targetPath.resolve(fileBaseName))) {
 						CryptoUtil.write(out, publicKey);
 					}
-				} catch (ExecutionException ex) {
-					throw new IOException(ex);
+				} catch (Exception ex) {
+					//throw new IOException(ex);
+					log.log(ProgressLog.Level.WARNING, "failed downloading entry for: "+publicKeyInfoToDownload);
 				}
 			}
 		}
@@ -481,6 +484,7 @@ private static Options buildOptions() {
 		options.addOption(Option.builder().longOpt("non-recursive").desc("Do not scan direcories recursively").build());
 		options.addOption(Option.builder().longOpt("init-keys").desc("Ensure keys exist and exit").build());
 		options.addOption(Option.builder().longOpt("min-signatures").numberOfArgs(1).argName("count").desc("Minimum required signatures to download a cache entry").build());
+		options.addOption(Option.builder().longOpt("cache-target-dir").numberOfArgs(1).argName("dir").desc("Override default cache target directory").build());
 		return options;
 	}
 

dxvk-cache-pool-client/src/main/java/com/ignorelist/kassandra/dxvk/cache/pool/client/Configuration.java

@@ -21,7 +21,7 @@ public class Configuration {
 	public static final String WINE_PREFIX_SYMLINK="dxvk-cache-pool";
 	public static final Path CONFIG_SUBDIR=Paths.get("dxvk-cache-pool");
 
-	private String host="http://kassandra.ignorelist.com:16969/";
+	private String host="http://173.212.215.164:16969";
 	private Path cacheTargetPath;
 	private Path configurationPath;
 	private Path cacheReferencePath;
@@ -61,6 +61,10 @@ public synchronized Path getCacheTargetPath() throws IOException {
 		return cacheTargetPath;
 	}
 
+	public synchronized void setCacheTargetPath(Path cacheTargetPath) {
+		this.cacheTargetPath=cacheTargetPath;
+	}
+
 	public synchronized Path getConfigurationPath() throws IOException {
 		if (null==configurationPath) {
 			Path configHome=Util.getEnvPath("XDG_CONFIG_HOME");

dxvk-cache-pool-client/src/main/java/com/ignorelist/kassandra/dxvk/cache/pool/client/rest/JerseyClientFactory.java

@@ -21,8 +21,8 @@
  */
 final class JerseyClientFactory extends BasePooledObjectFactory<Client> {
 
-	private static final int CONNECT_TIMEOUT=1000;
-	private static final int READ_TIMEOUT=30000;
+	private static final int CONNECT_TIMEOUT=2000;
+	private static final int READ_TIMEOUT=120000;
 
 	@Override
 	public Client create() throws Exception {

dxvk-cache-pool-common/src/main/java/com/ignorelist/kassandra/dxvk/cache/pool/common/StateCacheIO.java

@@ -32,11 +32,12 @@
 public class StateCacheIO {
 
 	private static final Logger LOG=Logger.getLogger(StateCacheIO.class.getName());
+	private static final String MAGIC_BYTES="DXVK";
 
 	/**
 	 * Parse StateCache and its entries.
 	 *
-	 * Will set basename base on the passed Path
+	 * Will set basename based on the passed Path
 	 *
 	 * @param path
 	 * @return
@@ -74,20 +75,20 @@ public static StateCache parse(final InputStream inputStream) throws Unsupported
 		};
 		 */
 		byte[] magicBytes=new byte[4];
-		inputStream.read(magicBytes);
+		ByteStreams.readFully(inputStream, magicBytes);
 		final String magicString=new String(magicBytes, Charsets.US_ASCII);
-		if (!"DXVK".equals(magicString)) {
+		if (!MAGIC_BYTES.equals(magicString)) {
 			throw new UnsupportedOperationException("wrong header: "+magicString);
 		}
 		byte[] versionBytes=new byte[4];
-		inputStream.read(versionBytes);
+		ByteStreams.readFully(inputStream, versionBytes);
 		final int version=parseUnsignedInt(versionBytes);
 		if (!StateCacheHeaderInfo.getKnownVersions().contains(version)) {
-			LOG.log(Level.WARNING, "unknon version encountered: {0}", version);
+			LOG.log(Level.WARNING, "unknown version encountered: {0}", version);
 		}
 
 		byte[] entrySizeBytes=new byte[4];
-		inputStream.read(entrySizeBytes);
+		ByteStreams.readFully(inputStream, entrySizeBytes);
 		final int entrySize=parseUnsignedInt(entrySizeBytes);
 		if (entrySize<=1||entrySize>StateCacheHeaderInfo.ENTRY_SIZE_MAX||(StateCacheHeaderInfo.getKnownVersions().contains(version)&&StateCacheHeaderInfo.getEntrySize(version)!=entrySize)) {
 			throw new IllegalStateException("header corrupt? entry size: "+entrySize);
@@ -139,7 +140,7 @@ public static void write(final OutputStream out, StateCache cache) throws IOExce
 		if (entrySize<=1) {
 			throw new IllegalStateException("illegal entry size: "+entrySize);
 		}
-		out.write("DXVK".getBytes(Charsets.US_ASCII));
+		out.write(MAGIC_BYTES.getBytes(Charsets.US_ASCII));
 		out.write(toUnsignedIntBytes(version));
 		out.write(toUnsignedIntBytes(entrySize));
 		cache.getEntries().stream()

dxvk-cache-pool-common/src/main/java/com/ignorelist/kassandra/dxvk/cache/pool/common/crypto/PublicKey.java

@@ -9,7 +9,6 @@
 import java.io.Serializable;
 import java.util.Arrays;
 import javax.validation.constraints.NotNull;
-import javax.xml.bind.annotation.XmlAttribute;
 import javax.xml.bind.annotation.XmlElement;
 import javax.xml.bind.annotation.XmlRootElement;
 
@@ -51,7 +50,7 @@ public void setKey(byte[] key) {
 	}
 
 	@NotNull
-	@XmlAttribute(required=true)
+	@XmlElement(required=true)
 	public PublicKeyInfo getKeyInfo() {
 		return keyInfo;
 	}

dxvk-cache-pool-common/src/main/java/com/ignorelist/kassandra/dxvk/cache/pool/common/crypto/PublicKeyInfo.java

@@ -13,7 +13,7 @@
 import java.util.Arrays;
 import javax.validation.constraints.NotNull;
 import javax.validation.constraints.Size;
-import javax.xml.bind.annotation.XmlAttribute;
+import javax.xml.bind.annotation.XmlElement;
 import javax.xml.bind.annotation.XmlRootElement;
 
 /**
@@ -38,7 +38,7 @@ public PublicKeyInfo(PublicKey publicKey) {
 
 	@NotNull
 	@Size(min=32, max=32)
-	@XmlAttribute(required=true)
+	@XmlElement(required=true)
 	public byte[] getHash() {
 		return hash;
 	}

dxvk-cache-pool-common/src/main/java/com/ignorelist/kassandra/dxvk/cache/pool/common/crypto/Signature.java

@@ -8,7 +8,7 @@
 import java.io.Serializable;
 import java.util.Arrays;
 import javax.validation.constraints.NotNull;
-import javax.xml.bind.annotation.XmlAttribute;
+import javax.xml.bind.annotation.XmlElement;
 import javax.xml.bind.annotation.XmlRootElement;
 
 /**
@@ -28,7 +28,7 @@ public Signature(byte[] signature) {
 	}
 
 	@NotNull
-	@XmlAttribute(required=true)
+	@XmlElement(required=true)
 	public byte[] getSignature() {
 		return signature;
 	}

dxvk-cache-pool-common/src/main/java/com/ignorelist/kassandra/dxvk/cache/pool/common/model/StateCacheEntryInfo.java

@@ -11,7 +11,7 @@
 import java.util.Arrays;
 import javax.validation.constraints.NotNull;
 import javax.validation.constraints.Size;
-import javax.xml.bind.annotation.XmlAttribute;
+import javax.xml.bind.annotation.XmlElement;
 import javax.xml.bind.annotation.XmlRootElement;
 
 /**
@@ -32,7 +32,7 @@ public StateCacheEntryInfo(byte[] hash) {
 
 	@NotNull
 	@Size(min=32, max=32)
-	@XmlAttribute(required=true)
+	@XmlElement(required=true)
 	public byte[] getHash() {
 		return hash;
 	}

dxvk-cache-pool-server/README.md

@@ -0,0 +1,19 @@
+# DXVK cache pool server
+
+Server (users do not need this):
+- Centralized storage. Provides REST interface to access caches and signatures.
+
+## Usage
+
+The server requires Java >= 8.
+
+### Server
+```bash
+$ ./dxvk-cache-server -h
+usage: dvxk-cache-server [-h] [--port <port>] [--storage <path>]
+       [--versions <version>]
+ -h,--help                 show this help
+    --port <port>          Server port
+    --storage <path>       Storage path
+    --versions <version>   DXVK state cache versions to accept
+```