New false positives in 5.4.0 for functions that aren't ESLint rules

[JoshuaKGoldberg]

typescript-eslint/typescript-eslint#8574 attempts to upgrade eslint-plugin-eslint-plugin from 5.3.0 -> 5.4.0. It has the following new failures:

/home/runner/work/typescript-eslint/typescript-eslint/packages/eslint-plugin/tests/RuleTester.ts
  43:8  error  `meta.messages` must contain at least one violation message                    eslint-plugin/prefer-message-ids
  43:8  error  Rules should be declared using the object style                                eslint-plugin/prefer-object-rule
  43:8  error  `meta.type` is required (must be either `problem`, `suggestion`, or `layout`)  eslint-plugin/require-meta-type
  43:8  error  `meta.schema` is required (use [] if rule has no schema)                       eslint-plugin/require-meta-schema

/home/runner/work/typescript-eslint/typescript-eslint/packages/eslint-plugin/tests/rules/naming-convention/cases/createTestCases.ts
  85:8  error  `meta.messages` must contain at least one violation message                    eslint-plugin/prefer-message-ids
  85:8  error  Rules should be declared using the object style                                eslint-plugin/prefer-object-rule
  85:8  error  `meta.type` is required (must be either `problem`, `suggestion`, or `layout`)  eslint-plugin/require-meta-type
  85:8  error  `meta.schema` is required (use [] if rule has no schema)                       eslint-plugin/require-meta-schema

/home/runner/work/typescript-eslint/typescript-eslint/packages/eslint-plugin/tests/rules/prefer-optional-chain/base-cases.ts
  232:43  error  `meta.messages` must contain at least one violation message                    eslint-plugin/prefer-message-ids
  232:43  error  Rules should be declared using the object style                                eslint-plugin/prefer-object-rule
  232:43  error  `meta.type` is required (must be either `problem`, `suggestion`, or `layout`)  eslint-plugin/require-meta-type
  232:43  error  `meta.schema` is required (use [] if rule has no schema)                       eslint-plugin/require-meta-schema

None of those three flagged code constructs should be reported on:

• tests/RuleTester.ts: export function batchedSingleLineTests
• naming-convention/cases/createTestCases.ts: export function createTestCases(cases: Cases): void {
• prefer-optional-chain/base-cases.ts: export const BaseCases: BaseCaseCreator = ({

It looks like the detection for what is an ESLint rule might be overly permissive?

[aladdin-add]

it was introduced in #449. Possible options:

1. keep the behavior as is, and fix the docs to recommend only appling the config to eslint rules/tests.
2. revert the change, and reintroduce it behind an option.

I prefer the option 2. wdyt? @bmish @43081j

[bmish]

It looks like the new false positives are with functions? Can we update the new detection in #449 to avoid trying to detect function-style rules? Function-style rules are obsolete and are hard to distinguish compared to newer object-style rules. Note: I want to keep existing function-style rule detection, but just avoid adding new detection of function-style rules that #449 seems to have caused.

[43081j]

yeah i agree with @bmish

i think if we wanted to continue supporting function-style rules, we should actually fix forward instead of introducing an option/rolling back. i.e. we should change the detection of function rules to be smarter

but maybe now is the time to stop supporting those rules since they're very old syntax.