Add note on project status (#89)

3 files changed, 86 insertions, 5 deletions

diff --git a/README.md b/README.md
index 5cff3d9..b2adc21 100644
--- a/README.md
+++ b/README.md
@@ -21,10 +21,15 @@ page per `.bzl`file.
 * Stardoc [rule reference](docs/stardoc_rule.md).
 * How to [contribute to Stardoc](docs/contributing.md)
 
-## Skydoc deprecation
+## Project Status
+
+### Skydoc deprecation
 
 Stardoc is a replacement for the **deprecated** "Skydoc" documentation generator.
 
 See [Skydoc Deprecation](docs/skydoc_deprecation.md) for
 details on the deprecation and migration details.
 
+### Future plans
+
+See our [future plans](docs/future_plans.md) for refactoring Stardoc to be more consistent with how Bazel evaluates .bzl files, and what it means for maintenance of this project.
diff --git a/docs/contributing.md b/docs/contributing.md
index f6daece..3a41a41 100644
--- a/docs/contributing.md
+++ b/docs/contributing.md
@@ -1,12 +1,18 @@
-We welcome your contributions! To contribute to Stardoc, first see the official
-[contributing notice](../CONTRIBUTING.md), then feel free to fork the
+Before contributing, please see the note on our [current
+priorities](future_plans.md). In short, we're able to take bugfixes to critical
+features, but cannot guarantee that we'll be able to review new features in a
+timely manner.
+
+To contribute to Stardoc, first see the official [contributing
+notice](../CONTRIBUTING.md), then feel free to fork the
 [Stardoc](https://github.com/bazelbuild/stardoc) GitHub repository and start
 submitting pull requests.
 
 In general, we prefer contributions that fix bugs or add features (as opposed to
 purely stylistic, refactoring, or "cleanup" changes). Please check with us by
-opening a [GitHub Issue](https://github.com/bazelbuild/stardoc/issues) or emailing the
-[bazel-dev](https://groups.google.com/forum/#!forum/bazel-dev) mailing list.
+opening a [GitHub Issue](https://github.com/bazelbuild/stardoc/issues) or
+emailing the [bazel-dev](https://groups.google.com/forum/#!forum/bazel-dev)
+mailing list.
 
 ## Stardoc code structure
 
diff --git a/docs/future_plans.md b/docs/future_plans.md
new file mode 100644
index 0000000..cbca65d
--- /dev/null
+++ b/docs/future_plans.md
@@ -0,0 +1,70 @@
+**Last updated:** January 2021.
+
+**Summary:** We'll be redesigning how Stardoc works, and deprioritizing feature
+requests and minor bugs until that work is complete (targeting 2021 Q2).
+
+## Technical motivation
+
+Stardoc is currently the recommended tool for generating documentation of
+Starlark rules. It [replaces](skydoc_deprecation.md) *Sky*doc, the previous
+tool, which worked by evaluating .bzl files in a Python interpreter, using
+fake versions of functions from Bazel's Build Language.
+
+Mocking is an inherently problematic approach for two reasons:
+
+1. It creates a maintenance burden for the tool maintainer (us). We have to
+ensure that the mocked definitions stay up-to-date as Bazel changes. These
+include not just `rule()` and `provider()`, but also a number of other symbols
+that don't directly affect documentation but still require stubs.
+
+2. It puts a constraint on the user: All of their documented .bzl files, as
+well as all of the .bzl dependencies they transitively load, must be
+compatible with the mock evaluation. This means users must be vigilant about
+writing Starlark code that lies in the intersection of what is understood as
+valid by Bazel and by Stardoc.
+
+The Python-based Skydoc experienced an extreme version of this problem because
+it didn't even treat .bzl files as being written in the Starlark language.
+However, the Java-based Stardoc still uses mocking -- not of the Starlark
+language, but of Bazel's Build Language functions.
+
+In addition, Stardoc's mocking approach tightly integrates it with Bazel.
+Indeed much of its source code lives inside the bazelbuild/bazel repository.
+This makes refactoring and evolving the Bazel source code more difficult.
+
+While the Starlark language has a specification and several implementations,
+the Build Language is more complicated and has only one accurate
+implementation: Bazel. Any tooling that operates on BUILD and .bzl files must
+carefully consider whether it is feasible to ask Bazel for the authoritative
+information. The alternative, falling back on simulation, not only produces
+less accurate results, but ties our hands as we try to improve Bazel.
+
+## Our plans
+
+We will rewrite the part of Stardoc that extracts documentation information from
+.bzl files, so that instead of using mocking to pseudo-evaluate individual .bzl
+files, it performs a real Bazel evaluation of the workspace. Think of how `bazel
+query` is used to dump out information from Bazel's loading phase about the
+target dependency graph. Now imagine that it's extended to also dump out the
+rules and providers declared in the .bzl files used by a build, and that this
+dump also includes their docstrings.
+
+This approach intersects other work we are doing to simplify and better specify
+Bazel's loading phase, so that users have access to all sorts of information
+that was previously not readily available.
+
+Note that this new internal approach does not necessarily have to mean the
+user's workflow changes. You could still write a target in your BUILD file to
+say exactly what content you want documented and how you'd like it formatted.
+The rendering logic may very well continue to live outside of Bazel, in the
+bazelbuild/stardoc repository.
+
+## Prioritization
+
+We're aiming to explore this design in more concrete detail in 2021 Q1, and
+implement it in Q2.
+
+In the meantime, *we will not be focusing on improvements to the current
+implementation of Stardoc*, even the formatting parts which might remain the
+same. We still commit to keeping Stardoc working for its existing essential use
+cases.