diff options
| -rw-r--r-- | docs/release-notes.md | 2 | ||||
| -rw-r--r-- | docs/technical/web.md | 69 | ||||
| -rw-r--r-- | src/SMAPI.Web/Framework/Clients/Pastebin/PasteInfo.cs | 2 | ||||
| -rw-r--r-- | src/SMAPI.Web/Framework/Clients/Pastebin/SavePasteResult.cs | 15 | ||||
| -rw-r--r-- | src/SMAPI.Web/Framework/ConfigModels/MongoDbConfig.cs | 6 | ||||
| -rw-r--r-- | src/SMAPI.Web/SMAPI.Web.csproj | 2 | ||||
| -rw-r--r-- | src/SMAPI.Web/Startup.cs | 25 | ||||
| -rw-r--r-- | src/SMAPI.Web/Views/Index/Privacy.cshtml | 8 | ||||
| -rw-r--r-- | src/SMAPI.Web/appsettings.Development.json | 6 | ||||
| -rw-r--r-- | src/SMAPI.Web/wwwroot/Content/js/json-validator.js | 6 |
10 files changed, 81 insertions, 60 deletions
diff --git a/docs/release-notes.md b/docs/release-notes.md index bbe08c13..bae1cb98 100644 --- a/docs/release-notes.md +++ b/docs/release-notes.md @@ -27,7 +27,7 @@ * Fixed asset propagation for `Characters\Farmer\farmer_girl_base_bald`. * For SMAPI developers: - * You can now run local environments without configuring Amazon, Azure, and Pastebin accounts. + * You can now run local environments without configuring Amazon, Azure, MongoDB, and Pastebin accounts. ## 3.0.1 Released 02 December 2019 for Stardew Valley 1.4.0.1. diff --git a/docs/technical/web.md b/docs/technical/web.md index 97e0704a..697ec280 100644 --- a/docs/technical/web.md +++ b/docs/technical/web.md @@ -10,17 +10,21 @@ and update check API. * [Short URLs](#short-urls) * [For SMAPI developers](#for-smapi-developers) * [Local development](#local-development) - * [Deploying to Amazon Beanstalk](#deploying-to-amazon-beanstalk) + * [Production environment](#production-environment) ## Log parser -The log parser provides a web UI for uploading, parsing, and sharing SMAPI logs. The logs are -persisted in a compressed form to Pastebin. The log parser lives at https://smapi.io/log. +The log parser at https://smapi.io/log provides a web UI for uploading, parsing, and sharing SMAPI +logs. + +The logs are saved in a compressed form to Amazon Blob storage for 30 days. ## JSON validator ### Overview -The JSON validator provides a web UI for uploading and sharing JSON files, and validating them as -plain JSON or against a predefined format like `manifest.json` or Content Patcher's `content.json`. -The JSON validator lives at https://smapi.io/json. +The JSON validator at https://smapi.io/json provides a web UI for uploading and sharing JSON files, +and validating them as plain JSON or against a predefined format like `manifest.json` or Content +Patcher's `content.json`. + +The logs are saved in a compressed form to Amazon Blob storage for 30 days. ### Schema file format Schema files are defined in `wwwroot/schemas` using the [JSON Schema](https://json-schema.org/) @@ -336,43 +340,46 @@ short url | → | target page A local environment lets you run a complete copy of the web project (including cache database) on your machine, with no external dependencies aside from the actual mod sites. -Initial setup: - -1. [Install MongoDB](https://docs.mongodb.com/manual/administration/install-community/) and add its - `bin` folder to the system PATH. -2. Create a local folder for the MongoDB data (e.g. `C:\dev\smapi-cache`). -3. Enter your credentials in the `appsettings.Development.json` file. You can leave the MongoDB - credentials as-is to use the default local instance; see the next section for the other settings. - -To launch the environment: -1. Launch MongoDB from a terminal (change the data path if applicable): - ```sh - mongod --dbpath C:\dev\smapi-cache - ``` -2. Launch `SMAPI.Web` from Visual Studio to run a local version of the site. - <small>(Local URLs will use HTTP instead of HTTPS.)</small> +1. Enter the Nexus credentials in `appsettings.Development.json` . You can leave the other + credentials empty to default to fetching data anonymously, and storing data in-memory and + on disk. +2. Launch `SMAPI.Web` from Visual Studio to run a local version of the site. ### Production environment A production environment includes the web servers and cache database hosted online for public -access. This section assumes you're creating a new production environment from scratch (not using -the official live environment). +access. + +This section assumes you're creating a new environment on Azure, but the app isn't tied to any +Azure services. If you want to host it on a different site, you'll need to adjust the instructions +accordingly. Initial setup: -1. Launch an empty MongoDB server (e.g. using [MongoDB Atlas](https://www.mongodb.com/cloud/atlas)). -2. Create an AWS Beanstalk .NET environment with these environment properties: +1. Launch an empty MongoDB server (e.g. using [MongoDB Atlas](https://www.mongodb.com/cloud/atlas)) + for mod data. +2. Create an Azure Blob storage account for uploaded files. +3. Create an Azure App Services environment running the latest .NET Core on Linux or Windows. +4. Add these application settings in the new App Services environment: property name | description ------------------------------- | ----------------- - `LogParser:PastebinDevKey` | The [Pastebin developer key](https://pastebin.com/api#1) used to authenticate with the Pastebin API. - `LogParser:PastebinUserKey` | The [Pastebin user key](https://pastebin.com/api#8) used to authenticate with the Pastebin API. Can be left blank to post anonymously. - `ModUpdateCheck:GitHubPassword` | The password with which to authenticate to GitHub when fetching release info. - `ModUpdateCheck:GitHubUsername` | The username with which to authenticate to GitHub when fetching release info. + `ApiClients.AzureBlobConnectionString` | The connection string for the Azure Blob storage account created in step 2. + `ApiClients.GitHubUsername`<br />`ApiClients.GitHubPassword` | The login credentials for the GitHub account with which to fetch release info. If these are omitted, GitHub will impose much stricter rate limits. + `ApiClients:NexusApiKey` | The [Nexus API authentication key](https://github.com/Pathoschild/FluentNexus#init-a-client). `MongoDB:Host` | The hostname for the MongoDB instance. `MongoDB:Username` | The login username for the MongoDB instance. `MongoDB:Password` | The login password for the MongoDB instance. - `MongoDB:Database` | The database name (e.g. `smapi` in production or `smapi-edge` in testing environments). + `MongoDB:Database` | The MongoDB database name (e.g. `smapi` in production or `smapi-edge` in testing environments). + + Optional settings: + + property name | description + ------------------------------- | ----------------- + `BackgroundServices:Enabled` | Set to `true` to enable background processes like fetching data from the wiki, or false to disable them. + `Site:BetaEnabled` | Set to `true` to show a separate download button if there's a beta version of SMAPI in its GitHub releases. + `Site:BetaBlurb` | If `Site:BetaEnabled` is true and there's a beta version of SMAPI in its GitHub releases, this is shown on the beta download button as explanatory subtext. + `Site:SupporterList` | A list of Patreon supports to credit on the download page. To deploy updates: -1. Deploy the web project using [AWS Toolkit for Visual Studio](https://aws.amazon.com/visualstudio/). +1. [Deploy the web project from Visual Studio](https://docs.microsoft.com/en-us/visualstudio/deployment/quickstart-deploy-to-azure). 2. If the MongoDB schema changed, delete the MongoDB database. (It'll be recreated automatically.) diff --git a/src/SMAPI.Web/Framework/Clients/Pastebin/PasteInfo.cs b/src/SMAPI.Web/Framework/Clients/Pastebin/PasteInfo.cs index 1ef3ef12..813ea115 100644 --- a/src/SMAPI.Web/Framework/Clients/Pastebin/PasteInfo.cs +++ b/src/SMAPI.Web/Framework/Clients/Pastebin/PasteInfo.cs @@ -1,5 +1,3 @@ -using System; - namespace StardewModdingAPI.Web.Framework.Clients.Pastebin { /// <summary>The response for a get-paste request.</summary> diff --git a/src/SMAPI.Web/Framework/Clients/Pastebin/SavePasteResult.cs b/src/SMAPI.Web/Framework/Clients/Pastebin/SavePasteResult.cs deleted file mode 100644 index 89dab697..00000000 --- a/src/SMAPI.Web/Framework/Clients/Pastebin/SavePasteResult.cs +++ /dev/null @@ -1,15 +0,0 @@ -namespace StardewModdingAPI.Web.Framework.Clients.Pastebin -{ - /// <summary>The response for a save-log request.</summary> - internal class SavePasteResult - { - /// <summary>Whether the log was successfully saved.</summary> - public bool Success { get; set; } - - /// <summary>The saved paste ID (if <see cref="Success"/> is <c>true</c>).</summary> - public string ID { get; set; } - - /// <summary>The error message (if saving failed).</summary> - public string Error { get; set; } - } -} diff --git a/src/SMAPI.Web/Framework/ConfigModels/MongoDbConfig.cs b/src/SMAPI.Web/Framework/ConfigModels/MongoDbConfig.cs index 3c508300..e2e18477 100644 --- a/src/SMAPI.Web/Framework/ConfigModels/MongoDbConfig.cs +++ b/src/SMAPI.Web/Framework/ConfigModels/MongoDbConfig.cs @@ -24,6 +24,12 @@ namespace StardewModdingAPI.Web.Framework.ConfigModels /********* ** Public method *********/ + /// <summary>Get whether a MongoDB instance is configured.</summary> + public bool IsConfigured() + { + return !string.IsNullOrWhiteSpace(this.Host); + } + /// <summary>Get the MongoDB connection string.</summary> public string GetConnectionString() { diff --git a/src/SMAPI.Web/SMAPI.Web.csproj b/src/SMAPI.Web/SMAPI.Web.csproj index 22f5e975..504254cd 100644 --- a/src/SMAPI.Web/SMAPI.Web.csproj +++ b/src/SMAPI.Web/SMAPI.Web.csproj @@ -14,6 +14,7 @@ <ItemGroup> <PackageReference Include="Azure.Storage.Blobs" Version="12.1.0" /> <PackageReference Include="Hangfire.AspNetCore" Version="1.7.7" /> + <PackageReference Include="Hangfire.MemoryStorage" Version="1.6.3" /> <PackageReference Include="Hangfire.Mongo" Version="0.6.5" /> <PackageReference Include="HtmlAgilityPack" Version="1.11.16" /> <PackageReference Include="Humanizer.Core" Version="2.7.9" /> @@ -23,6 +24,7 @@ <PackageReference Include="Microsoft.AspNetCore.Mvc" Version="2.2.0" /> <PackageReference Include="Microsoft.AspNetCore.Rewrite" Version="2.2.0" /> <PackageReference Include="Microsoft.AspNetCore.StaticFiles" Version="2.2.0" /> + <PackageReference Include="Mongo2Go" Version="2.2.12" /> <PackageReference Include="MongoDB.Driver" Version="2.9.3" /> <PackageReference Include="Newtonsoft.Json.Schema" Version="3.0.11" /> <PackageReference Include="Pathoschild.FluentNexus" Version="0.8.0" /> diff --git a/src/SMAPI.Web/Startup.cs b/src/SMAPI.Web/Startup.cs index 07ee0c9e..338cd2d5 100644 --- a/src/SMAPI.Web/Startup.cs +++ b/src/SMAPI.Web/Startup.cs @@ -1,5 +1,7 @@ +using System; using System.Collections.Generic; using Hangfire; +using Hangfire.MemoryStorage; using Hangfire.Mongo; using Microsoft.AspNetCore.Builder; using Microsoft.AspNetCore.Hosting; @@ -8,6 +10,7 @@ using Microsoft.AspNetCore.Routing; using Microsoft.Extensions.Configuration; using Microsoft.Extensions.DependencyInjection; using Microsoft.Extensions.Options; +using Mongo2Go; using MongoDB.Bson.Serialization; using MongoDB.Driver; using Newtonsoft.Json; @@ -89,10 +92,20 @@ namespace StardewModdingAPI.Web } // init MongoDB + services.AddSingleton<MongoDbRunner>(serv => !mongoConfig.IsConfigured() + ? MongoDbRunner.Start() + : throw new InvalidOperationException("The MongoDB connection is configured, so the local development version should not be used.") + ); services.AddSingleton<IMongoDatabase>(serv => { + // get connection string + string connectionString = mongoConfig.IsConfigured() + ? mongoConfig.GetConnectionString() + : serv.GetRequiredService<MongoDbRunner>().ConnectionString; + + // get client BsonSerializer.RegisterSerializer(new UtcDateTimeOffsetSerializer()); - return new MongoClient(mongoConfig.GetConnectionString()).GetDatabase(mongoConfig.Database); + return new MongoClient(connectionString).GetDatabase(mongoConfig.Database); }); services.AddSingleton<IModCacheRepository>(serv => new ModCacheRepository(serv.GetRequiredService<IMongoDatabase>())); services.AddSingleton<IWikiCacheRepository>(serv => new WikiCacheRepository(serv.GetRequiredService<IMongoDatabase>())); @@ -104,12 +117,18 @@ namespace StardewModdingAPI.Web config .SetDataCompatibilityLevel(CompatibilityLevel.Version_170) .UseSimpleAssemblyNameTypeSerializer() - .UseRecommendedSerializerSettings() - .UseMongoStorage(mongoConfig.GetConnectionString(), $"{mongoConfig.Database}-hangfire", new MongoStorageOptions + .UseRecommendedSerializerSettings(); + + if (mongoConfig.IsConfigured()) + { + config.UseMongoStorage(mongoConfig.GetConnectionString(), $"{mongoConfig.Database}-hangfire", new MongoStorageOptions { MigrationOptions = new MongoMigrationOptions(MongoMigrationStrategy.Drop), CheckConnection = false // error on startup takes down entire process }); + } + else + config.UseMemoryStorage(); }); // init API clients diff --git a/src/SMAPI.Web/Views/Index/Privacy.cshtml b/src/SMAPI.Web/Views/Index/Privacy.cshtml index 7327de3d..fd78f908 100644 --- a/src/SMAPI.Web/Views/Index/Privacy.cshtml +++ b/src/SMAPI.Web/Views/Index/Privacy.cshtml @@ -22,10 +22,10 @@ <h2>Data collected and transmitted</h2> <h3 id="web-logging">Web logging</h3> -<p>This website and SMAPI's web API are hosted by Amazon Web Services. Their servers may automatically collect diagnostics like your IP address, but this information is not visible to SMAPI's web application or developers. For more information, see the <a href="https://aws.amazon.com/privacy/">Amazon Privacy Notice</a>.</p> +<p>This website and SMAPI's web API are hosted on Microsoft Azure. Their servers may automatically collect diagnostics like your IP address, but this information is not visible to SMAPI's web apps or its developers. For more information, see the <a href="https://azure.microsoft.com/en-ca/support/legal/">Microsoft Azure legal resources</a>.</p> <h3>Update checks</h3> -<p>SMAPI notifies you when there's a new version of SMAPI or your mods available. To do so, it sends your game/SMAPI/mod versions and platform type to its web API. No personal information is stored by the web application, but see <em><a href="#web-logging">web logging</a></em>.</p> +<p>SMAPI notifies you when there's a new version of SMAPI or your mods available. To do so, it sends basic metadata like your game/SMAPI/mod versions and platform type to its web API. No personal information is stored by the web app.</p> <p>You can disable update checks, and no information will be transmitted to the web API. To do so:</p> <ol> @@ -34,8 +34,8 @@ <li>change <code>"CheckForUpdates": true</code> to <code>"CheckForUpdates": false</code>.</li> </ol> -<h3>Log parser</h3> -<p>The <a href="https://smapi.io/log">log parser page</a> lets you store a log file for analysis and sharing. The log data is stored indefinitely in an obfuscated form as unlisted pastes in <a href="https://pastebin.com/">Pastebin</a>. No personal information is stored by the log parser beyond what you choose to upload, but see <em><a href="#web-logging">web logging</a></em> and the <a href="https://pastebin.com/doc_privacy_statement">Pastebin Privacy Statement</a>.</p> +<h3>Log parser and JSON validator</h3> +<p>The <a href="https://smapi.io/log">log parser</a> and <a href="https://smapi.io/json">JSON validator</a> let you upload files to analyze and share with other users. The log data is stored for 30 days in an obfuscated form in a private Microsoft Azure Blob storage account. No personal information is stored by the log parser beyond what you choose to upload as part of those files.</p> <h3>Multiplayer sync</h3> <p>As part of its multiplayer API, SMAPI transmits basic context to players you connect to (mainly your OS, SMAPI version, game version, and installed mods). This is used to enable multiplayer features like inter-mod messages, compatibility checks, etc. Although this information is normally hidden from players, it may be visible due to mods or configuration changes.</p> diff --git a/src/SMAPI.Web/appsettings.Development.json b/src/SMAPI.Web/appsettings.Development.json index 3c2001ef..a6e48c69 100644 --- a/src/SMAPI.Web/appsettings.Development.json +++ b/src/SMAPI.Web/appsettings.Development.json @@ -18,9 +18,13 @@ }, "MongoDB": { - "Host": "localhost", + "Host": null, "Username": null, "Password": null, "Database": "smapi-edge" + }, + + "BackgroundServices": { + "Enabled": true } } diff --git a/src/SMAPI.Web/wwwroot/Content/js/json-validator.js b/src/SMAPI.Web/wwwroot/Content/js/json-validator.js index 401efbee..72b8192b 100644 --- a/src/SMAPI.Web/wwwroot/Content/js/json-validator.js +++ b/src/SMAPI.Web/wwwroot/Content/js/json-validator.js @@ -71,9 +71,9 @@ smapi.LineNumberRange = function (maxLines) { /** * UI logic for the JSON validator page. * @param {string} urlFormat The URL format for a file, with $schemaName and $id placeholders. - * @param {string} pasteID The Pastebin paste ID for the content being viewed, if any. + * @param {string} fileId The file ID for the content being viewed, if any. */ -smapi.jsonValidator = function (urlFormat, pasteID) { +smapi.jsonValidator = function (urlFormat, fileId) { /** * The original content element. */ @@ -138,7 +138,7 @@ smapi.jsonValidator = function (urlFormat, pasteID) { // change format $("#output #format").on("change", function() { var schemaName = $(this).val(); - location.href = urlFormat.replace("$schemaName", schemaName).replace("$id", pasteID); + location.href = urlFormat.replace("$schemaName", schemaName).replace("$id", fileId); }); // upload form |