2025-05-09 19:03 AEST

View Issue Details Jump to Notes ]
IDProjectCategoryView StatusLast Update
0000273mercuryBugpublic2013-11-19 12:50
Reporterony 
Assigned Topbone 
PrioritylowSeverityminorReproducibilitysometimes
StatusclosedResolutionfixed 
Product Version 
Target VersionFixed in Version 
Summary0000273: race condition in Mmakefiles of deep_profiler and slice
DescriptionDuring build with -j9 from master branch I wasn't able to succeed few times because of permission deny for copying in $(MDBCOMP_MODULES).
Steps To ReproduceAt master branch bootstrap from existing compiler and use multiple jobs
make PARALLEL=-j9
Additional InformationLooks like the reason for that is race condition from rule that fired twice in parallel for same files and thus had scenario:
1. [thread1] chmod a+w feedback.m
2. [thread1] cp ../mdbcomp/feedback.m .
3. [thread2] chmod a+w feedback.m
4. [thread1] chmod a-w feedback.m
5. [thread2] cp ../mdbcomp/feedback.m . # fails on permission deny

Refactoring process of pulling in modules from $(MDBCOMP_DIR) should solve this issue.
TagsNo tags attached.
Attached Files
  • patch file icon mercury-scm-fix-bug273-race-cond.-in-slice-and-deep_profile.patch (7,000 bytes) 2013-01-06 13:10 -
    From 02f884a6b55de4ce0cfc34bf363c3cd341733363 Mon Sep 17 00:00:00 2001
    From: Nikolay Orlyuk <virkony@gmail.com>
    Date: Sat, 5 Jan 2013 20:12:45 +0200
    Subject: [PATCH 4/4] fix bug273 (race cond. in slice and deep_profile)
    
    Removed mdbcomp modules copy-in. Instead just include mdbcomp in search
    subdirs.
    
    See http://bugs.mercury.csse.unimelb.edu.au/view.php?id=273
    ---
     deep_profiler/DEEP_FLAGS.in |  3 +++
     deep_profiler/Mmakefile     | 39 +++++----------------------------------
     slice/Mmakefile             | 44 +++++++-------------------------------------
     slice/SLICE_FLAGS.in        |  3 +++
     4 files changed, 18 insertions(+), 71 deletions(-)
    
    diff --git a/deep_profiler/DEEP_FLAGS.in b/deep_profiler/DEEP_FLAGS.in
    index d73d03a..cda923a 100644
    --- a/deep_profiler/DEEP_FLAGS.in
    +++ b/deep_profiler/DEEP_FLAGS.in
    @@ -6,6 +6,7 @@
     -I../library
     -I../browser
     -I../ssdb
    +-I../mdbcomp
     --c-include-directory ../boehm_gc
     --c-include-directory ../boehm_gc/include
     --c-include-directory ../runtime
    @@ -15,6 +16,8 @@
     --c-include-directory ../browser/Mercury/mihs
     --c-include-directory ../ssdb
     --c-include-directory ../ssdb/Mercury/mihs
    +--c-include-directory ../mdbcomp
    +--c-include-directory ../mdbcomp/Mercury/mihs
     --c-include-directory ../trace
     --csharp-flag -keyfile:../mercury.snk
     --no-java-classpath
    diff --git a/deep_profiler/Mmakefile b/deep_profiler/Mmakefile
    index c43fc09..2bdf85c 100644
    --- a/deep_profiler/Mmakefile
    +++ b/deep_profiler/Mmakefile
    @@ -54,23 +54,7 @@ else
     	INSTALL=nothing
     endif
     
    -VPATH = $(LIBRARY_DIR) $(SSDB_DIR)
    -
    -#-----------------------------------------------------------------------------#
    -
    -MDBCOMP_MODULES = \
    -	feedback.automatic_parallelism.m \
    -	feedback.m \
    -	mdbcomp.m \
    -	mdbcomp.goal_path.m \
    -	prim_data.m \
    -	program_representation.m \
    -	rtti_access.m \
    -	shared_utilities.m \
    -	slice_and_dice.m \
    -	trace_counts.m
    -
    -MDBCOMP_ORIG_MODULES = $(patsubst %,$(MDBCOMP_DIR)/%,$(MDBCOMP_MODULES))
    +VPATH = $(LIBRARY_DIR) $(MDBCOMP_DIR) $(SSDB_DIR)
     
     #-----------------------------------------------------------------------------#
     
    @@ -107,9 +91,9 @@ endif
     nothing:
     
     .PHONY: depend
    -depend:	$(MDBCOMP_MODULES) $(DEPEND)
    +depend:	$(DEPEND)
     
    -$(DEPEND): DEEP_FLAGS $(MDBCOMP_MODULES) Mercury.modules
    +$(DEPEND): DEEP_FLAGS Mercury.modules
     
     # This directory contains source files for which the module
     # name doesn't match the file name, so smart recompilation
    @@ -119,19 +103,7 @@ Mercury.modules: DEEP_FLAGS
     	$(MC) $(ALL_GRADEFLAGS) $(ALL_MCFLAGS) -f *.m
     
     .PHONY: all
    -all:	$(MDBCOMP_MODULES) $(ALL_DEEP_MODULES) $(TAGS_FILE_EXISTS)
    -
    -# We need to start by turning write permission on for each copied file
    -# in case some exist, but we need to ignore errors in case some don't exist.
    -# The exit 0 is to prevent make itself from printing a message about the
    -# (ignored) failure of an action.
    -#
    -# We could modify the action here to copy only the changed files.
    -
    -$(MDBCOMP_MODULES): $(MDBCOMP_ORIG_MODULES)
    -	-@chmod a+w $(MDBCOMP_MODULES) > /dev/null 2>&1; exit 0
    -	cp $(MDBCOMP_ORIG_MODULES) .
    -	@chmod a-w $(MDBCOMP_MODULES)
    +all:	$(ALL_DEEP_MODULES) $(TAGS_FILE_EXISTS)
     
     #-----------------------------------------------------------------------------#
     
    @@ -231,8 +203,7 @@ cs: $(mdprof_procrep.cs) $(cs_subdir)mdprof_procrep_init.c
     #-----------------------------------------------------------------------------#
     
     realclean_local:
    -	rm -f .deep_tags tags DEEP_FLAGS DEEP_FLAGS.date \
    -		$(MDBCOMP_MODULES) mdbcomp.*.err
    +	rm -f .deep_tags tags DEEP_FLAGS DEEP_FLAGS.date
     
     #-----------------------------------------------------------------------------#
     
    diff --git a/slice/Mmakefile b/slice/Mmakefile
    index cdfca10..7ef40e4 100644
    --- a/slice/Mmakefile
    +++ b/slice/Mmakefile
    @@ -42,23 +42,7 @@ INTS	= $(patsubst %,%.ints,$(MERCURY_MAIN_MODULES))
     INT3S	= $(patsubst %,%.int3s,$(MERCURY_MAIN_MODULES))
     CHECKS	= $(patsubst %,%.check,$(MERCURY_MAIN_MODULES))
     
    -VPATH = $(LIBRARY_DIR) $(SSDB_DIR)
    -
    -#-----------------------------------------------------------------------------#
    -
    -MDBCOMP_MODULES = \
    -	feedback.automatic_parallelism.m \
    -	feedback.m \
    -	mdbcomp.m \
    -	mdbcomp.goal_path.m \
    -	prim_data.m \
    -	program_representation.m \
    -	rtti_access.m \
    -	shared_utilities.m \
    -	slice_and_dice.m \
    -	trace_counts.m
    -
    -MDBCOMP_ORIG_MODULES = $(patsubst %,$(MDBCOMP_DIR)/%,$(MDBCOMP_MODULES))
    +VPATH = $(LIBRARY_DIR) $(MDBCOMP_DIR) $(SSDB_DIR)
     
     #-----------------------------------------------------------------------------#
     
    @@ -82,26 +66,12 @@ endif
     #-----------------------------------------------------------------------------#
     
     .PHONY: depend
    -depend:	$(MDBCOMP_MODULES) $(DEPENDS)
    +depend:	$(DEPENDS)
     
    -$(DEPENDS): SLICE_FLAGS $(MDBCOMP_MODULES)
    +$(DEPENDS): SLICE_FLAGS
     
     .PHONY: all
    -all:	$(MDBCOMP_MODULES) $(MERCURY_MAIN_MODULES) $(TAGS_FILE_EXISTS)
    -
    -#-----------------------------------------------------------------------------#
    -#
    -# We need to start by turning write permission on for each copied file
    -# in case some exist, but we need to ignore errors in case some don't exist.
    -# The exit 0 is to prevent make itself from printing a message about the
    -# (ignored) failure of an action.
    -#
    -# We could modify the action here to copy only the changed files.
    -
    -$(MDBCOMP_MODULES): $(MDBCOMP_ORIG_MODULES)
    -	-@chmod a+w $(MDBCOMP_MODULES) > /dev/null 2>&1; exit 0
    -	cp $(MDBCOMP_ORIG_MODULES) .
    -	@chmod a-w $(MDBCOMP_MODULES)
    +all:	$(MERCURY_MAIN_MODULES) $(TAGS_FILE_EXISTS)
     
     #-----------------------------------------------------------------------------#
     
    @@ -157,7 +127,8 @@ ints:	$(INTS)
     tags:	.slice_tags
     
     .slice_tags: $(MTAGS) $(mslice.ms) $(mdice.ms) $(mtc_union.ms) $(mcov.ms) \
    -		$(mtc_diff.ms) $(wildcard $(LIBRARY_DIR)/*.m)
    +		$(mtc_diff.ms) $(wildcard $(LIBRARY_DIR)/*.m) \
    +		$(wildcard $(MDBCOMP_DIR)/*.m)
     	$(MTAGS) $(mslice.ms) $(mdice.ms) $(mtc_union.ms) $(mcov.ms) \
     		$(mtc_diff.ms) $(LIBRARY_DIR)/*.m
     	@touch .slice_tags
    @@ -194,8 +165,7 @@ ils:	$(mslice.ils) $(mdice.ils) $(mtc_union.ils) $(mcov.ils) $(mtc_diff.ils)
     #-----------------------------------------------------------------------------#
     
     realclean_local:
    -	rm -f tags SLICE_FLAGS SLICE_FLAGS.date \
    -		$(MDBCOMP_MODULES) mdbcomp.*.err
    +	rm -f tags SLICE_FLAGS SLICE_FLAGS.date
     
     #-----------------------------------------------------------------------------#
     
    diff --git a/slice/SLICE_FLAGS.in b/slice/SLICE_FLAGS.in
    index 2a4321c..78bde9e 100644
    --- a/slice/SLICE_FLAGS.in
    +++ b/slice/SLICE_FLAGS.in
    @@ -6,6 +6,7 @@
     -I../library
     -I../browser
     -I../ssdb
    +-I../mdbcomp
     --c-include-directory ../boehm_gc
     --c-include-directory ../boehm_gc/include
     --c-include-directory ../runtime
    @@ -15,6 +16,8 @@
     --c-include-directory ../browser/Mercury/mihs
     --c-include-directory ../ssdb
     --c-include-directory ../ssdb/Mercury/mihs
    +--c-include-directory ../mdbcomp
    +--c-include-directory ../mdbcomp/Mercury/mihs
     --c-include-directory ../trace
     --csharp-flag -keyfile:../mercury.snk
     --no-java-classpath
    -- 
    1.8.0.3
    
    

-Relationships
+Relationships

-Notes

~0000491

juliensf (administrator)

Copying the files instead of referring to the mdbcomp directory is (clearly) deliberate. The reason
for doing so to avoid stack overflows in some of the profiling / slice tools when the entire Mercury system
is built with tracing enabled. (The proposed patch would break this and make testing the Mercury
compiler in some configurations impossible.)

The race condition can be avoided by building the dependencies *before* doing the main build, e.g.
do

    $ mmake depend
    $ mmake -j9

~0000582

pbone (administrator)

I fixed this back in January and forgot to close the bug then.
+Notes

-Issue History
Date Modified Username Field Change
2013-01-06 10:42 ony New Issue
2013-01-06 13:10 ony File Added: mercury-scm-fix-bug273-race-cond.-in-slice-and-deep_profile.patch
2013-01-07 11:43 juliensf Note Added: 0000491
2013-11-19 12:49 pbone Note Added: 0000582
2013-11-19 12:50 pbone Status new => closed
2013-11-19 12:50 pbone Assigned To => pbone
2013-11-19 12:50 pbone Resolution open => fixed
+Issue History