c++: Attempt to implement C++26 P3034R1 - Module Declarations Shouldn't be Macros [PR114461]

This is an attempt to implement the https://wg21.link/p3034r1 paper,
but I'm afraid the wording in the paper is bad for multiple reasons.
I think I understand the intent, that the module name and partition
if any shouldn't come from macros so that they can be scanned for
without preprocessing, but on the other side doesn't want to disable
macro expansion in pp-module altogether, because e.g. the optional
attribute in module-declaration would be nice to come from macros
as which exact attribute is needed might need to be decided based on
preprocessor checks.
The paper added https://eel.is/c++draft/cpp.module#2
which uses partly the wording from https://eel.is/c++draft/cpp.module#1

The first issue I see is that using that "defined as an object-like macro"
from there means IMHO something very different in those 2 paragraphs.
As per https://eel.is/c++draft/cpp.pre#7.sentence-1 preprocessing tokens
in preprocessing directives aren't subject to macro expansion unless
otherwise stated, and so the export and module tokens aren't expanded
and so the requirement that they aren't defined as an object-like macro
makes perfect sense.  The problem with the new paragraph is that
https://eel.is/c++draft/cpp.module#3.sentence-1 says that the rest of
the tokens are macro expanded and after macro expansion none of the
tokens can be defined as an object-like macro, if they would be, they'd
be expanded to that.  So, I think either the wording needs to change
such that not all preprocessing tokens after module are macro expanded,
only those which are after the pp-module-name and if any pp-module-partition
tokens, or all tokens after module are macro expanded but none of the tokens in
pp-module-name and pp-module-partition if any must come from macro
expansion.  The patch below implements it as if the former would be
specified (but see later), so essentially scans the preprocessing tokens
after module without expansion, if the first one is an identifier, it
disables expansion for it and then if followed by . or : expects another
such identifier (again with disabled expansion), but stops after second
: is seen.

Second issue is that while the global-module-fragment start is fine, matches
the syntax of the new paragraph where the pp-tokens[opt] aren't present,
there is also private-module-fragment in the syntax where module is
followed by : private ; and in that case the colon doesn't match the
pp-module-name grammar and appears now to be invalid.  I think the
https://eel.is/c++draft/cpp.module#2
paragraph needs to change so that it allows also that pp-tokens of
a pp-module may also be : pp-tokens[opt] (and in that case, I think
the colon shouldn't come from a macro and private and/or ; can).

Third issue is that there are too many pp-tokens in
https://eel.is/c++draft/cpp.module , one is all the tokens between
module keyword and the semicolon and one is the optional extra tokens
after pp-module-partition (if any, if missing, after pp-module).
Perhaps introducing some other non-terminal would help talking about it?
So in "where the pp-tokens (if any) shall not begin with a ( preprocessing
token" it isn't obvious which pp-tokens it is talking about (my assumption
is the latter) and also whether ( can't appear there just before macro
expansion or also after expansion.  The patch expects only before expansion,
so
 #define F ();
 export module foo F
would be valid during preprocessing but obviously invalid during
compilation, but
 #define foo(n) n;
 export module foo (3)
would be invalid already during preprocessing.

The last issue applies only if the first issue is resolved to allow
expansion of tokens after : if first token, or after pp-module-partition
if present or after pp-module-name if present.  When non-preprocessing
scanner sees
 export module foo.bar:baz.qux;
it knows nothing can come from preprocessing macros and is ok, but if it
sees
 export module foo.bar:baz qux
then it can't know whether it will be
 export module foo.bar:baz;
or
 export module foo.bar:baz [[]];
or
 export module foo.bar:baz.freddy.garply;
because qux could be validly a macro, which expands to ; or [[]];
or .freddy.garply; etc.  So, either the non-preprocessing scanner would
need to note it as possible export of foo.bar:baz* module partitions
and preprocess if it needs to know the details or just compile, or if that
is not ok, the wording would need to rule out that the expansion of (the
second) pp-tokens if any can't start with . or : (colon would be only
problematic if it isn't present in the tokens before it already).
So, if e.g. defining qux above to . whatever is invalid, then the scanner
can rely it sees the whole module name and partition.

The patch below implements what is above described as the first variant
of the first issue resolution, i.e. disables expansion of as many tokens
as could be in the valid module name and module partition syntax, but
as soon as it e.g. sees two adjacent identifiers, the second one can be
macro expanded.  If it is macro expanded though, the expansion can't
start with . or :, and if it expands to nothing, tokens after it (whether
they come from macro expansion or not) can't start with . or :.
So, effectively:
 #define SEMI ;
 export module SEMI
used to be valid and isn't anymore,
 #define FOO bar
 export module FOO;
isn't valid,
 #define COLON :
 export module COLON private;
isn't valid,
 #define BAR baz
 export module foo.bar:baz.qux.BAR;
isn't valid,
 #define BAZ .qux
 export module foo BAZ;
isn't valid,
 #define FREDDY :garply
 export module foo FREDDY;
isn't valid,
while
 #define QUX [[]]
 export module foo QUX;
or
 #define GARPLY private
 module : GARPLY;
etc. is.

2024-11-01  Jakub Jelinek  <jakub@redhat.com>

	PR c++/114461
libcpp/
	* include/cpplib.h: Implement C++26 P3034R1
	- Module Declarations Shouldn’t be Macros (or more precisely
	its expected intent).
	(NO_DOT_COLON): Define.
	* internal.h (struct cpp_reader): Add diagnose_dot_colon_from_macro_p
	member.
	* lex.cc (cpp_maybe_module_directive): For pp-module, if
	module keyword is followed by CPP_NAME, ensure all CPP_NAME
	tokens possibly matching module name and module partition
	syntax aren't expanded and aren't defined as object-like macros.
	Verify first token after that doesn't start with open paren.
	If the next token after module name/partition is CPP_NAME defined
	as macro, set NO_DOT_COLON flag on it.
	* macro.cc (cpp_get_token_1): Set
	pfile->diagnose_dot_colon_from_macro_p if token to be expanded has
	NO_DOT_COLON bit set in flags.  Before returning, if
	pfile->diagnose_dot_colon_from_macro_p is true and not returning
	CPP_PADDING or CPP_COMMENT and not during macro expansion preparation,
	set pfile->diagnose_dot_colon_from_macro_p to false and diagnose
	if returning CPP_DOT or CPP_COLON.
gcc/testsuite/
	* g++.dg/modules/cpp-7.C: New test.
	* g++.dg/modules/cpp-8.C: New test.
	* g++.dg/modules/cpp-9.C: New test.
	* g++.dg/modules/cpp-10.C: New test.
	* g++.dg/modules/cpp-11.C: New test.
	* g++.dg/modules/cpp-12.C: New test.
	* g++.dg/modules/cpp-13.C: New test.
	* g++.dg/modules/cpp-14.C: New test.
	* g++.dg/modules/cpp-15.C: New test.
	* g++.dg/modules/cpp-16.C: New test.
	* g++.dg/modules/cpp-17.C: New test.
	* g++.dg/modules/cpp-18.C: New test.
	* g++.dg/modules/cpp-19.C: New test.
	* g++.dg/modules/cpp-20.C: New test.
	* g++.dg/modules/pmp-4.C: New test.
	* g++.dg/modules/pmp-5.C: New test.
	* g++.dg/modules/pmp-6.C: New test.
	* g++.dg/modules/token-6.C: New test.
	* g++.dg/modules/token-7.C: New test.
	* g++.dg/modules/token-8.C: New test.
	* g++.dg/modules/token-9.C: New test.
	* g++.dg/modules/token-10.C: New test.
	* g++.dg/modules/token-11.C: New test.
	* g++.dg/modules/token-12.C: New test.
	* g++.dg/modules/token-13.C: New test.
	* g++.dg/modules/token-14.C: New test.
	* g++.dg/modules/token-15.C: New test.
	* g++.dg/modules/token-16.C: New test.
	* g++.dg/modules/dir-only-3.C: Expect an error.
	* g++.dg/modules/dir-only-4.C: Expect an error.
	* g++.dg/modules/dir-only-5.C: New test.
	* g++.dg/modules/atom-preamble-2_a.C: In export module malcolm;
	replace malcolm with kevin.  Don't define malcolm macro.
	* g++.dg/modules/atom-preamble-4.C: Expect an error.
	* g++.dg/modules/atom-preamble-5.C: New test.
This commit is contained in:
Jakub Jelinek 2024-11-01 19:42:28 +01:00 committed by Jakub Jelinek
parent 79a75b1f55
commit 1ae24f7e0b
38 changed files with 328 additions and 4 deletions

View File

@ -1,6 +1,5 @@
// { dg-additional-options "-fmodules-ts" }
#define malcolm kevin
export module malcolm;
export module kevin;
// { dg-module-cmi kevin }
export class X;

View File

@ -3,3 +3,5 @@
export module NAME(bob)
// { dg-error "module name followed by '\\\('" "" { target *-*-* } .-2 }
// { dg-error "expected ';' before '\\\(' token" "" { target *-*-* } .-3 }

View File

@ -0,0 +1,5 @@
// { dg-additional-options "-fmodules-ts" }
#define NAME(X) ;
export module bob NAME(bob)

View File

@ -0,0 +1,7 @@
// { dg-do preprocess }
// { dg-additional-options "-fmodules-ts" }
#define bob fred;
export module foo:bob; // { dg-error "module partition 'bob' cannot be an object-like macro" }
int i;

View File

@ -0,0 +1,7 @@
// { dg-do preprocess }
// { dg-additional-options "-fmodules-ts" }
#define bob fred;
export module foo:bar.bob; // { dg-error "module partition 'bob' cannot be an object-like macro" }
int i;

View File

@ -0,0 +1,7 @@
// { dg-do preprocess }
// { dg-additional-options "-fmodules-ts" }
#define baz .qux // { dg-error "'\\\.' in module name or partition comes from or after macro expansion" }
export module foo:bar baz;
int i;

View File

@ -0,0 +1,7 @@
// { dg-do preprocess }
// { dg-additional-options "-fmodules-ts" }
#define baz :qux.garply // { dg-error "':' in module name or partition comes from or after macro expansion" }
export module foo.bar baz;
int i;

View File

@ -0,0 +1,7 @@
// { dg-do preprocess }
// { dg-additional-options "-fmodules-ts" }
#define baz [[]]
export module foo.bar baz;
int i;

View File

@ -0,0 +1,8 @@
// { dg-do preprocess }
// { dg-additional-options "-fmodules-ts" }
#define baz
#define qux
export module foo.bar baz . garply qux; // { dg-error "'\\\.' in module name or partition comes from or after macro expansion" }
int i;

View File

@ -0,0 +1,8 @@
// { dg-do preprocess }
// { dg-additional-options "-fmodules-ts" }
#define baz
#define qux
export module foo.bar baz : garply qux; // { dg-error "':' in module name or partition comes from or after macro expansion" }
int i;

View File

@ -0,0 +1,7 @@
// { dg-do preprocess }
// { dg-additional-options "-fmodules-ts" }
#define baz(x) x qux
export module foo:bar baz(.); // { dg-error "'\\\.' in module name or partition comes from or after macro expansion" }
int i;

View File

@ -0,0 +1,7 @@
// { dg-do preprocess }
// { dg-additional-options "-fmodules-ts" }
#define baz(x) x qux.garply
export module foo.bar baz(:); // { dg-error "':' in module name or partition comes from or after macro expansion" }
int i;

View File

@ -0,0 +1,8 @@
// { dg-do preprocess }
// { dg-additional-options "-fmodules-ts" }
#define baz(x)
#define qux(x)
export module foo.bar baz(.) . garply qux(:); // { dg-error "'\\\.' in module name or partition comes from or after macro expansion" }
int i;

View File

@ -0,0 +1,8 @@
// { dg-do preprocess }
// { dg-additional-options "-fmodules-ts" }
#define baz(x)
#define qux(x)
export module foo.bar baz(:) : garply qux(.); // { dg-error "':' in module name or partition comes from or after macro expansion" }
int i;

View File

@ -0,0 +1,8 @@
// { dg-do preprocess }
// { dg-additional-options "-fmodules-ts" }
#define NAME(X) X;
export module NAME(bob) // { dg-error "module name followed by '\\\('" }
int i;

View File

@ -0,0 +1,7 @@
// { dg-do preprocess }
// { dg-additional-options "-fmodules-ts" }
#define bob fred;
export module bob; // { dg-error "module name 'bob' cannot be an object-like macro" }
int i;

View File

@ -0,0 +1,7 @@
// { dg-do preprocess }
// { dg-additional-options "-fmodules-ts" }
#define bob fred;
export module foo.bob; // { dg-error "module name 'bob' cannot be an object-like macro" }
int i;

View File

@ -7,10 +7,12 @@
# 32 "<command-line>" 2
# 1 "dir-only-3.C"
// { dg-additional-options {-fmodules-ts -fpreprocessed -fdirectives-only} }
// { dg-module-cmi foo }
// { dg-module-cmi !foo }
module;
#define foo baz
export module foo;
// { dg-error "module name 'foo' cannot be an object-like macro" "" { target *-*-* } 5 }
// { dg-prune-output "not writing module" }
class import {};

View File

@ -2,7 +2,7 @@
// { dg-module-cmi !foo }
module;
#define foo baz
export module foo;
export module foo; // { dg-error "module name 'foo' cannot be an object-like macro" }
class import {};

View File

@ -0,0 +1,9 @@
// { dg-additional-options {-fmodules-ts -fpreprocessed -fdirectives-only} }
// { dg-module-cmi !baz }
module;
#define foo baz
export module baz;
class import {};
import x; // { dg-error "post-module-declaration" }

View File

@ -0,0 +1,9 @@
// { dg-additional-options -fmodules-ts }
export module bob;
// { dg-module-cmi bob }
#define PRIVATE private
module :PRIVATE; // { dg-message "sorry, unimplemented: private module fragment" }
int i;

View File

@ -0,0 +1,9 @@
// { dg-additional-options -fmodules-ts }
export module bob;
// { dg-module-cmi bob }
#define PRIVATE_SEMI private ;
module :PRIVATE_SEMI // { dg-message "sorry, unimplemented: private module fragment" }
int i;

View File

@ -0,0 +1,9 @@
// { dg-additional-options -fmodules-ts }
export module bob;
// { dg-module-cmi bob }
#define SEMI ;
module :private SEMI // { dg-message "sorry, unimplemented: private module fragment" }
int i;

View File

@ -0,0 +1,6 @@
// { dg-additional-options "-fmodules-ts" }
#define semi ;
export module foo.bar:baz.bob semi
// { dg-module-cmi foo.bar:baz.bob }

View File

@ -0,0 +1,6 @@
// { dg-additional-options "-fmodules-ts" }
#define attr [[]]
export module foo.bar:baz.bob attr ;
// { dg-module-cmi foo.bar:baz.bob }

View File

@ -0,0 +1,6 @@
// { dg-additional-options "-fmodules-ts" }
#define bob() fred
export module bob;
// { dg-module-cmi bob }

View File

@ -0,0 +1,6 @@
// { dg-additional-options "-fmodules-ts" }
#define bob() fred
export module foo.bar.bob;
// { dg-module-cmi foo.bar.bob }

View File

@ -0,0 +1,6 @@
// { dg-additional-options "-fmodules-ts" }
#define bob(n) fred
export module foo.bar:bob;
// { dg-module-cmi foo.bar:bob }

View File

@ -0,0 +1,6 @@
// { dg-additional-options "-fmodules-ts" }
#define bob() fred
export module foo.bar:baz.bob;
// { dg-module-cmi foo.bar:baz.bob }

View File

@ -0,0 +1,9 @@
// { dg-additional-options "-fmodules-ts" }
#define bob() fred
export module foo.bar:baz.bob ();
// { dg-error "module partition followed by '\\\('" "" { target *-*-* } .-1 }
// { dg-error "expected" "" { target *-*-* } .-2 }
// { dg-module-cmi !foo.bar:baz.bob }
// { dg-module-cmi !foo.bar:baz.fred }

View File

@ -0,0 +1,9 @@
// { dg-additional-options "-fmodules-ts" }
#define bob fred
export module bob;
// { dg-error "module name 'bob' cannot be an object-like macro" "" { target *-*-* } .-1 }
// { dg-module-cmi !bob }
// { dg-module-cmi !fred }
// { dg-prune-output "not writing module" }

View File

@ -0,0 +1,9 @@
// { dg-additional-options "-fmodules-ts" }
#define bob fred
export module foo.bar.bob;
// { dg-error "module name 'bob' cannot be an object-like macro" "" { target *-*-* } .-1 }
// { dg-module-cmi !foo.bar.bob }
// { dg-module-cmi !foo.bar.fred }
// { dg-prune-output "not writing module" }

View File

@ -0,0 +1,9 @@
// { dg-additional-options "-fmodules-ts" }
#define bob fred
export module foo.bar:bob;
// { dg-error "module partition 'bob' cannot be an object-like macro" "" { target *-*-* } .-1 }
// { dg-module-cmi !foo.bar:bob }
// { dg-module-cmi !foo.bar:fred }
// { dg-prune-output "not writing module" }

View File

@ -0,0 +1,9 @@
// { dg-additional-options "-fmodules-ts" }
#define garply fred
export module foo.bar:baz.garply;
// { dg-error "module partition 'garply' cannot be an object-like macro" "" { target *-*-* } .-1 }
// { dg-module-cmi !foo.bar:baz.garply }
// { dg-module-cmi !foo.bar:baz.fred }
// { dg-prune-output "not writing module" }

View File

@ -204,6 +204,9 @@ struct GTY(()) cpp_string {
#define PURE_ZERO (1 << 7) /* Single 0 digit, used by the C++ frontend,
set in c-lex.cc. */
#define COLON_SCOPE PURE_ZERO /* Adjacent colons in C < 23. */
#define NO_DOT_COLON PURE_ZERO /* Set on CPP_NAME tokens whose expansion
shouldn't start with CPP_DOT or CPP_COLON
after optional CPP_PADDING. */
#define SP_DIGRAPH (1 << 8) /* # or ## token was a digraph. */
#define SP_PREV_WHITE (1 << 9) /* If whitespace before a ##
operator, or before this token

View File

@ -468,6 +468,10 @@ struct cpp_reader
one. */
bool about_to_expand_macro_p;
/* True if the preprocessor should diagnose CPP_DOT or CPP_COLON
tokens as the first ones coming from macro expansion. */
bool diagnose_dot_colon_from_macro_p;
/* Search paths for include files. */
struct cpp_dir *quote_include; /* "" */
struct cpp_dir *bracket_include; /* <> */

View File

@ -3604,6 +3604,78 @@ cpp_maybe_module_directive (cpp_reader *pfile, cpp_token *result)
/* Maybe tell the tokenizer we expect a header-name down the
road. */
pfile->state.directive_file_token = header_count;
/* According to P3034R1, pp-module-name and pp-module-partition tokens
if any shouldn't be macro expanded and identifiers shouldn't be
defined as object-like macro. */
if (!header_count && peek->type == CPP_NAME)
{
int state = 0;
do
{
cpp_token *tok = peek;
if (tok->type == CPP_NAME)
{
cpp_hashnode *node = tok->val.node.node;
/* Don't attempt to expand the token. */
tok->flags |= NO_EXPAND;
if (_cpp_defined_macro_p (node)
&& _cpp_maybe_notify_macro_use (pfile, node,
tok->src_loc)
&& !cpp_fun_like_macro_p (node))
{
if (state == 0)
cpp_error_with_line (pfile, CPP_DL_ERROR,
tok->src_loc, 0,
"module name %qs cannot "
"be an object-like macro",
NODE_NAME (node));
else
cpp_error_with_line (pfile, CPP_DL_ERROR,
tok->src_loc, 0,
"module partition %qs cannot "
"be an object-like macro",
NODE_NAME (node));
}
}
peek = _cpp_lex_direct (pfile);
backup++;
if (tok->type == CPP_NAME)
{
if (peek->type == CPP_DOT)
continue;
else if (peek->type == CPP_COLON && state == 0)
{
++state;
continue;
}
else if (peek->type == CPP_OPEN_PAREN)
{
if (state == 0)
cpp_error_with_line (pfile, CPP_DL_ERROR,
peek->src_loc, 0,
"module name followed by %<(%>");
else
cpp_error_with_line (pfile, CPP_DL_ERROR,
peek->src_loc, 0,
"module partition followed by "
"%<(%>");
break;
}
else if (peek->type == CPP_NAME
&& _cpp_defined_macro_p (peek->val.node.node))
{
peek->flags |= NO_DOT_COLON;
break;
}
else
break;
}
else if (peek->type != CPP_NAME)
break;
}
while (true);
}
}
else
{

View File

@ -3076,6 +3076,9 @@ cpp_get_token_1 (cpp_reader *pfile, location_t *location)
if (pfile->state.prevent_expansion)
break;
if ((result->flags & NO_DOT_COLON) != 0)
pfile->diagnose_dot_colon_from_macro_p = true;
/* Conditional macros require that a predicate be evaluated
first. */
if ((node->flags & NODE_CONDITIONAL) != 0)
@ -3226,6 +3229,20 @@ cpp_get_token_1 (cpp_reader *pfile, location_t *location)
}
}
if (pfile->diagnose_dot_colon_from_macro_p
&& !pfile->about_to_expand_macro_p
&& result->type != CPP_PADDING
&& result->type != CPP_COMMENT)
{
if (result->type == CPP_DOT || result->type == CPP_COLON)
cpp_error_with_line (pfile, CPP_DL_ERROR,
result->src_loc, 0,
"%qc in module name or partition "
"comes from or after macro expansion",
result->type == CPP_DOT ? '.' : ':');
pfile->diagnose_dot_colon_from_macro_p = false;
}
return result;
}