Revise documentation (#2728)

Co-authored-by: Sarah Haggarty <sarahhaggarty@users.noreply.github.com>

3 files changed, 484 insertions, 0 deletions

diff --git a/mkdocs/src/doc/docs/developer_guide/architecture/data_model/documentables.md b/mkdocs/src/doc/docs/developer_guide/architecture/data_model/documentables.md
new file mode 100644
index 00000000..5264553d
--- /dev/null
+++ b/mkdocs/src/doc/docs/developer_guide/architecture/data_model/documentables.md
@@ -0,0 +1,245 @@
+# Documentables Model
+
+Documentables represent data that is parsed from sources. Think of this data model as of something that could be
+seen or produced by a compiler frontend, it's not far off from the truth.
+
+By default, documentables are parsed from `Descriptor` (for `Kotlin`)
+and [Psi](https://plugins.jetbrains.com/docs/intellij/psi.html)
+(for `Java`) models. Code-wise, you can have a look at following classes:
+
+* `DefaultDescriptorToDocumentableTranslator` - responsible for `Kotlin` -> `Documentable` mapping
+* `DefaultPsiToDocumentableTranslator` - responsible for `Java` -> `Documentable` mapping
+
+Upon creation, it's a collection of trees, each with `DModule` as root.
+
+Take some arbitrary `Kotlin` source code that is located within the same module:
+
+```kotlin
+// Package 1
+class Clazz(val property: String) {
+    fun function(parameter: String) {}
+}
+
+fun topLevelFunction() {}
+
+// Package 2
+enum class Enum { }
+
+val topLevelProperty: String
+```
+
+This would be represented roughly as the following `Documentable` tree:
+
+```mermaid
+flowchart TD
+    DModule --> firstPackage[DPackage]
+    firstPackage --> DClass
+    firstPackage --> toplevelfunction[DFunction] 
+    DClass --> DProperty
+    DClass --> DFunction
+    DFunction --> DParameter
+    DModule --> secondPackage[DPackage]
+    secondPackage --> DEnum
+    secondPackage --> secondPackageProperty[DProperty]
+```
+
+At later stages of transformation, all trees are folded into one (by `DocumentableMerger`).
+
+## Documentable
+
+The main building block of documentables model is `Documentable` class. It's the base class for all more specific types
+that represent elements of parsed sources with mostly self-explanatory names (`DFunction`, `DPackage`, `DProperty`, etc)
+.
+`DClasslike` is the base class for class-like documentables such as `DClass`, `DEnum`, `DAnnotation`, etc.
+
+The contents of each documentable normally represent what you would see in source code. For instance, if you open
+`DClass`, you should find that it contains references to functions, properties, companion object, constructors and so
+on.
+`DEnum` should have references to enum entries, and `DPackage` can have references to both classlikes and top-level
+functions and properties (`Kotlin`-specific).
+
+Here's an example of a documentable:
+
+```kotlin
+data class DClass(
+    val dri: DRI,
+    val name: String,
+    val constructors: List<DFunction>,
+    val functions: List<DFunction>,
+    val properties: List<DProperty>,
+    val classlikes: List<DClasslike>,
+    val sources: SourceSetDependent<DocumentableSource>,
+    val visibility: SourceSetDependent<Visibility>,
+    val companion: DObject?,
+    val generics: List<DTypeParameter>,
+    val supertypes: SourceSetDependent<List<TypeConstructorWithKind>>,
+    val documentation: SourceSetDependent<DocumentationNode>,
+    val expectPresentInSet: DokkaSourceSet?,
+    val modifier: SourceSetDependent<Modifier>,
+    val sourceSets: Set<DokkaSourceSet>,
+    val isExpectActual: Boolean,
+    val extra: PropertyContainer<DClass> = PropertyContainer.empty()
+) : DClasslike(), WithAbstraction, WithCompanion, WithConstructors,
+    WithGenerics, WithSupertypes, WithExtraProperties<DClass>
+```
+
+___
+
+There are three non-documentable classes that important for this model:
+
+* `DRI`
+* `SourceSetDependent`
+* `ExtraProperty`.
+
+### DRI
+
+`DRI` stans for _Dokka Resource Identifier_ - a unique value that identifies a specific `Documentable`.
+All references and relations between documentables (other than direct ownership) are described using `DRI`.
+
+For example, `DFunction` with a parameter of type `Foo` has only `Foo`'s `DRI`, not the actual reference
+to `Foo`'s `Documentable` object.
+
+#### Example
+
+For an example of how a `DRI` can look like, let's take the `limitedParallelism` function from `kotlinx.coroutines`:
+
+```kotlin
+package kotlinx.coroutines
+
+import ...
+        
+public abstract class MainCoroutineDispatcher : CoroutineDispatcher() {
+    
+    override fun limitedParallelism(parallelism: Int): CoroutineDispatcher {
+        ...
+    }
+}
+```
+
+If we were to re-create the DRI of this function in code, it would look something like this:
+
+```kotlin
+DRI(
+    packageName = "kotlinx.coroutines",
+    classNames = "MainCoroutineDispatcher",
+    callable = Callable(
+        name = "limitedParallelism",
+        receiver = null,
+        params = listOf(
+            TypeConstructor(
+                fullyQualifiedName = "kotlin.Int",
+                params = emptyList()
+            )
+        )
+    ),
+    target = PointingToDeclaration,
+    extra = null
+)
+```
+
+If you format it as `String`, it would look like this:
+
+```
+kotlinx.coroutines/MainCoroutineDispatcher/limitedParallelism/#kotlin.Int/PointingToDeclaration/
+```
+
+### SourceSetDependent
+
+`SourceSetDependent` helps handling multiplatform data by associating platform-specific data (declared with either
+`expect` or `actual` modifier) with particular 
+[source sets](https://kotlinlang.org/docs/multiplatform-discover-project.html#source-sets).
+
+This comes in handy if `expect`/`actual` declarations differ. For instance, the default value for `actual` might differ
+from that declared in `expect`, or code comments written for `expect` might be different from what's written
+for `actual`.
+
+Under the hood, it's a `typealias` to a `Map`:
+
+```kotlin
+typealias SourceSetDependent<T> = Map<DokkaSourceSet, T>
+```
+
+### ExtraProperty
+
+`ExtraProperty` is used to store any additional information that falls outside of the regular model. It is highly
+recommended to use extras to provide any additional information when creating custom Dokka plugins.
+
+This element is a bit more complex, so you can read more about how to use it
+[in a separate section](extra.md).
+
+___
+
+## Documentation model
+
+Documentation model is used alongside Documentables to store data obtained by parsing
+code comments (such as `KDoc`/`Javadoc`).
+
+### DocTag
+
+`DocTag` describes a specific documentation syntax element.
+
+It's universal across source languages. For instance, DocTag `B` is the same for `**bold**` in `Kotlin` and
+`<b>bold</b>` in `Java`.
+
+However, some `DocTag` elements are specific to a certain language, there are many such examples for `Java`
+because it allows HTML tags inside `Javadoc` comments, some of which are simply not possible to reproduce with `Markdown`.
+
+`DocTag` elements can be deeply nested with other `DocTag` children elements.
+
+Examples:
+
+```kotlin
+data class H1(
+    override val children: List<DocTag> = emptyList(),
+    override val params: Map<String, String> = emptyMap()
+) : DocTag()
+
+data class H2(
+    override val children: List<DocTag> = emptyList(),
+    override val params: Map<String, String> = emptyMap()
+) : DocTag()
+
+data class Strikethrough(
+    override val children: List<DocTag> = emptyList(),
+    override val params: Map<String, String> = emptyMap()
+) : DocTag()
+
+data class Strong(
+    override val children: List<DocTag> = emptyList(),
+    override val params: Map<String, String> = emptyMap()
+) : DocTag()
+
+data class CodeBlock(
+    override val children: List<DocTag> = emptyList(),
+    override val params: Map<String, String> = emptyMap()
+) : Code()
+
+```
+
+### TagWrapper
+
+`TagWrapper` describes the whole comment description or a specific comment tag.
+For example: `@see` / `@author` / `@return`.
+
+Since each such section may contain formatted text inside of it, each `TagWrapper` has `DocTag` children.
+
+```kotlin
+/**
+ * @author **Ben Affleck*
+ * @return nothing, except _sometimes_ it may throw an [Error]
+ */
+fun foo() {}
+```
+
+### DocumentationNode
+
+`DocumentationNode` acts as a container for multiple `TagWrapper` elements for a specific `Documentable`, usually
+used like this:
+
+```kotlin
+data class DFunction(
+    ...
+    val documentation: SourceSetDependent<DocumentationNode>,
+    ...
+)
+```
diff --git a/mkdocs/src/doc/docs/developer_guide/architecture/data_model/extra.md b/mkdocs/src/doc/docs/developer_guide/architecture/data_model/extra.md
new file mode 100644
index 00000000..0abbc70e
--- /dev/null
+++ b/mkdocs/src/doc/docs/developer_guide/architecture/data_model/extra.md
@@ -0,0 +1,99 @@
+# Extra
+
+## Introduction
+
+`ExtraProperty` classes are used both by [Documentable](documentables.md) and [Content](page_content.md#content-model)
+models.
+
+Source code for `ExtraProperty`:
+
+```kotlin
+interface ExtraProperty<in C : Any> {
+    interface Key<in C : Any, T : Any> {
+        fun mergeStrategyFor(left: T, right: T): MergeStrategy<C> = MergeStrategy.Fail {
+            throw NotImplementedError("Property merging for $this is not implemented")
+        }
+    }
+
+    val key: Key<C, *>
+}
+```
+
+To declare a new extra, you need to implement `ExtraProperty` interface. It is advised to use following pattern
+when declaring new extras:
+
+```kotlin
+data class CustomExtra(
+    [any data relevant to your extra], 
+    [any data relevant to your extra] 
+): ExtraProperty<Documentable> {
+    override val key: CustomExtra.Key<Documentable, *> = CustomExtra
+    companion object : CustomExtra.Key<Documentable, CustomExtra>
+}
+```
+
+Merge strategy (`mergeStrategyFor` method) for extras is invoked during
+[merging](../extension_points/core_extensions.md#documentablemerger) if documentables from different 
+[source sets](https://kotlinlang.org/docs/multiplatform-discover-project.html#source-sets) each
+have their own `Extra` of the same type. 
+
+## PropertyContainer
+
+All extras for `ContentNode` and `Documentable` classes are stored in `PropertyContainer<C : Any>` class instances.
+
+```kotlin
+data class DFunction(
+    ...
+    override val extra: PropertyContainer<DFunction> = PropertyContainer.empty()
+    ...
+) : WithExtraProperties<DFunction>
+```
+
+`PropertyContainer` has a number of convenient functions for handling extras in a collection-like manner.
+
+The `C` generic class parameter limits the type of properties that can be stored in the container -  it must
+match generic `C` class parameter from `ExtraProperty` interface. This allows creating extra properties
+which can only be stored in a specific `Documentable`.
+
+## Usage example
+
+In following example we will create a `DFunction`-only property, store it and then retrieve its value:
+
+```kotlin
+data class CustomExtra(val customExtraValue: String) : ExtraProperty<DFunction> {
+    override val key: ExtraProperty.Key<Documentable, *> = CustomExtra
+    companion object: ExtraProperty.Key<Documentable, CustomExtra>
+}
+
+fun DFunction.withCustomExtraProperty(data: String): DFunction {
+    return this.copy(
+        extra = extra + CustomExtra(data)
+    )
+}
+
+fun DFunction.getCustomExtraPropertyValue(): String? {
+    return this.extra[CustomExtra]?.customExtraValue
+}
+```
+
+___
+
+You can also use extras as markers, without storing any data in them:
+
+```kotlin
+
+object MarkerExtra : ExtraProperty<Any>, ExtraProperty.Key<Any, MarkerExtra> {
+    override val key: ExtraProperty.Key<Any, *> = this
+}
+
+fun Documentable.markIfFunction(): Documentable {
+    return when(this) {
+        is DFunction -> this.copy(extra = extra + MarkerExtra)
+        else -> this
+    }
+}
+
+fun WithExtraProperties<Documentable>.isMarked(): Boolean {
+    return this.extra[MarkerExtra] != null
+}
+```
diff --git a/mkdocs/src/doc/docs/developer_guide/architecture/data_model/page_content.md b/mkdocs/src/doc/docs/developer_guide/architecture/data_model/page_content.md
new file mode 100644
index 00000000..54ded235
--- /dev/null
+++ b/mkdocs/src/doc/docs/developer_guide/architecture/data_model/page_content.md
@@ -0,0 +1,140 @@
+# Page / Content Model
+
+Even though `Page` and `Content` models reside on the same level (under `Page`), it's easier to view it as two different
+models altogether, even though `Content` is only used in conjunction with and inside `Page` model.
+
+## Page
+
+Page model represents the structure of documentation pages to be generated. During rendering, each page
+is processed separately, so one page corresponds to exactly one output file.
+
+Page model is independent of the final output format, in other words it's universal. Which extension the pages
+should be created as (`.html`, `.md`, etc) and how is up to the `Renderer`.
+
+Subclasses of `PageNode` represent different kinds of rendered pages, such as `ModulePage`, `PackagePage`,
+`ClasslikePage`, `MemberPage` (properties, functions), etc.
+
+The Page Model is a tree structure, with `RootPageNode` at the root.
+
+Here's an example of how an arbitrary `Page` tree might look like for a module with 3 packages, one of which contains
+a top level function, top level property and a class, inside which there's a function and a property:
+
+```mermaid
+flowchart TD
+    RootPageNode --> firstPackage[PackagePageNode]
+    RootPageNode --> secondPackage[PackagePageNode]
+    RootPageNode --> thirdPackage[PackagePageNode]
+    firstPackage --> firstPackageFirstMember[MemberPageNode - Function]
+    firstPackage --> firstPackageSecondMember[MemberPageNode - Property]
+    firstPackage ---> firstPackageClasslike[ClasslikePageNode - Class]
+    firstPackageClasslike --> firstPackageClasslikeFirstMember[MemberPageNode - Function]
+    firstPackageClasslike --> firstPackageClasslikeSecondMember[MemberPageNode - Property]
+    secondPackage --> etcOne[...]
+    thirdPackage --> etcTwo[...]
+```
+
+Almost all pages are derivatives of `ContentPage` - it's the type of `Page` that has `Content` on it.
+
+## Content Model
+
+Content model describes how the actual `Page` content is presented. The important thing to understand is that it's
+also output-format independent and is universal.
+
+Content model is essentially a set of building blocks that you can put together to represent some content.
+Have a look at subclasses of `ContentNode`: `ContentText`, `ContentList`, `ContentTable`, `ContentCodeBlock`, 
+`ContentHeader` and so on. You can group content together with `ContentGroup` - for instance,
+to wrap all children with some style.
+
+```kotlin
+// real example of composing content using `DocumentableContentBuilder` DSL
+orderedList {
+    item {
+        text("This list contains a nested table:")
+        table {
+            header {
+                text("Col1")
+                text("Col2")
+            }
+            row {
+                text("Text1")
+                text("Text2")
+            }
+        }
+    }
+    item {
+        group(styles = setOf(TextStyle.Bold)) {
+            text("This is bald")
+            text("This is also bald")
+        }
+    }
+}
+```
+
+It is then responsibility of `Renderer` (i.e specific output format) to render it the way it wants. 
+
+For instance, `HtmlRenderer` might render `ContentCodeBlock` as `<code>text</code>`, but `CommonmarkRenderer` might 
+render it using backticks.
+
+___
+
+### DCI
+
+Each node is identified by unique `DCI`, which stands for _Dokka Content Identifier_. `DCI` aggregates `DRI`s of all
+`Documentables` that make up a specific `ContentNode`.
+
+```kotlin
+data class DCI(val dri: Set<DRI>, val kind: Kind)
+```
+
+All references to other nodes (other than direct ownership) are described using `DCI`.
+
+### ContentKind
+
+`ContentKind` represents a grouping of content of one kind that can can be rendered as part of a composite
+page (one tab/block within a class's page, for instance).
+
+For instance, on the same page that describes a class you can have multiple sections (== `ContentKind`).
+One to describe functions, one to describe properties, another one to describe constructors and so on.
+
+### Styles
+
+Each `ContentNode` has `styles` property in case you want to incidate to `Renderer` that this content needs to be
+displayed in a certain way.
+
+```kotlin
+group(styles = setOf(TextStyle.Paragraph)) {
+    text("Text1", styles = setOf(TextStyle.Bold))
+    text("Text2", styles = setOf(TextStyle.Italic))
+}
+```
+
+It is then responsibility of `Renderer` (i.e specific output format) to render it the way it wants. For instance,
+`HtmlRenderer` might render `TextStyle.Bold` as `<b>text</b>`, but `CommonmarkRenderer` might render it as `**text**`.
+
+There's a number of existing styles that you can use, most of them are supported by `HtmlRenderer` out of the box:
+
+```kotlin
+// for code highlighting
+enum class TokenStyle : Style {
+    Keyword, Punctuation, Function, Operator, Annotation,
+    Number, String, Boolean, Constant, Builtin, ...
+}
+
+enum class TextStyle : Style {
+    Bold, Italic, Strong, Strikethrough, Paragraph, ...
+}
+
+enum class ContentStyle : Style {
+    TabbedContent, RunnableSample, Wrapped, Indented, ...
+}
+```
+
+### Extra
+
+`ExtraProperty` is used to store any additional information that falls outside of the regular model. It is highly
+recommended to use extras to provide any additional information when creating custom Dokka plugins.
+
+All `ExtraProperty` elements from `Documentable` model are propagated into `Content` model and are available
+for `Renderer`.
+
+This element is a bit complex, so you can read more about how to use it [in a separate section](extra.md).