Hello Francesco,
I and iii. I had did it before, however such arrays are limited to 40000 values; so I can't save entire table in arrays inside "First Call initializations". Well, maybe launching millions of primaries, randomly from these 40K be enough.
ii. Interesting point also,
iv. Therefore, just reinforcing it in other words: if I START with P primaries, use S spawns and C cycles, the real number of primaries will be C*S*P divided in S cores. Ergo, one should get similar statistics using "1E8 primaries, 1 cycle and 5 spawns"; "5E8 primaries 1 cycle and 1 spawn" or "2.5E8 primaries, 2 cycles and 1 spawn". Right?
v. Indeed, it's taking me a lot of time and we checked that when we change some regions for blackbodies we also have a factor 2 gain. However, since we have a long beamline and intend to simulate its end station soon, a two steps will save many repetitive simulations. This item, however, remind us another old doubt: do these statistical errors informed by USRBIN output mean the percentage of the value corresponding to the standard deviation? But, why is the maximum error equal to 100?
vi. even when I set "gfortran" in the config panel, "compile" uses g77
Best wishes
Marlon
-----Mensagem original-----
De: Francesco Cerutti [mailto:Francesco.Cerutti_at_cern.ch]
Enviada em: sábado, 12 de março de 2016 13:07
Para: Marlon Saveri Silva <marlon.saveri_at_lnls.br>
Cc: Mao, Stan <mao_at_SLAC.Stanford.EDU>; fluka-discuss_at_fluka.org
Assunto: Re: RES: RES: [fluka-discuss]: RES: Splitting FLUKA runs
Hallo Marlon,
there is still quite a bit of confusion.
i. The source routine has a first part intended as "First call initializations", under the condition IF ( LFIRST ).
There you have to read once and for all the whole content of your file, filling your arrays as you previously did through a DO cycle.
The rest of the routine is executed at the beginning of every primary history. There you have to sample one particle through the random choice of the array index, this one will be the particle initiating your second step primary history. So NPFLKA = NPFLKA + 1 should not be put inside a DO cycle.
ii. As I previously said, loading all million particles in the same primary history makes no sense at all. One has to run instead million primary histories of one particle each. Actually it would be more correct to identify the first step secondaries through their first step event number and then, in the second step source, to sample over the first step event numbers and load all secondaries belonging to the sampled event. So, if a first step event generated three secondaries, these should be loaded together in the same second step event. But this is not really a major issue in your case, since on average you have only half particle per first step event.
iii. The fact that the second step random choice may sample more than once the same particle and forget another one is perfectly consistent with the Monte Carlo philosophy. Of course in the second step one should *globally* run a number of histories exceeding the file size.
iv. The number of histories (or primaries) is hard coded in your input file (START card). Spawns and cycles just multiply it, as you realized.
Cycles do it by multiplying the CPU time, executing a cycle after the previous one on the same CPU. Spawns exploit the possible availability of multiple CPUs, i.e. contemporary runs.
v. A gain of a factor 2 in the CPU time needed to get a suitable convergence for the scored quantity, at first glance does not justify the considerable effort to set up and correctly handle a two step approach.
First, the same CPU time can be obtained by a single step simulation halving the statistics, with a limited penalty of sqrt(2) on the statistical error. Otherwise, a double CPU time does not look as a critical limitation. Second, one has also to consider the CPU time required by the first step as well as the human time and the complication of the simulation.
vi. Inside the Config panel of Flair one can set the compiler.
Best wishes
Francesco
**************************************************
Francesco Cerutti
CERN-EN/STI
CH-1211 Geneva 23
Switzerland
tel. ++41 22 7678962
fax ++41 22 7668854
On Fri, 11 Mar 2016, Marlon Saveri Silva wrote:
> Good afternoon all,
>
>
> About second step handling and editing source.f:
>
> I had imagined first solution (loading all particles each event) would return a more realistic simulation, because it ensures all particles from first step are going to second step. Using the second solution with FLRNDM(DUMMY), some particles could be launched more than others, since they are randomly chosen.
> I've tried this second way, however I come across a problem: how to move back the pointer of READ() function. For example, imagine the first random tells to Fortran "get the 45ª line", so, it'll take such line; in the next iteration, if the random raffle "13", Fortran will not read 13ª line, but 58ª and, after some iterations, READ() will reach the end of the file, because it does not start from the beginning. So, what I did was not using FLRNDM(DUMMY), but just READ() in order to read the next line of the file each iteration. This way, the number of primaries in the second step will be limited to the number of lines in the input file; but, at least it's "working" now.
>
> About need two steps:
>
> The answer whether I need two steps depends on how much time I'll be saving. It seems to me that It will be shorten by half. However this judgement takes me to another doubt (I hope the last for this subject):
>
> After all, when we work with N spawns and P primaries, isn't the simulation divided in P/N primaries by core?
>
> - First step using 1.E6 primaries with just 1 spawn and 1 cycle generated an output file with 472 306 particles crossing regions set at USERDUMP/mgdraw.
> - First step using 1.E6 primaries with 4 spawns and 1 cycle generated 4 output files with 472 306 + 472 701 + 474 139 + 472 352 = 1 891 498 particles. I expected it would generate 472306/4 = 10984 particles by core to be concatenated next. If each spawn uses all 1.E6 primaries, using spawn is useful "only" to get better statistic, not for speeding up, right? But isn't it already covered by the number of cycles?
>
> - Second step using 472306 particles took 1h10min (4 cycles, 5 spawns)
> - Second step using 1891498 particles took 4h50min (4 cycles, 5 spawns)
> - A single step simulation of this case using 1.E6 primaries took
> ~2h30min (4 cycles, 5 spawns);
>
> So, the way I see it:
> A. using two steps for this case means cutting simulation time by half.
> B. For scaling, I need multiply the final result by 472306/1.E6 = 0.47 no matter how many primaries I use.
> C. That simulation of a second step with 1891498 particles is, actually, equivalent to a 4.E6 primaries single simulation (but I didn’t test).
>
> About compilation inside Flair:
>
> I can compile new executables only when I use that codes on terminal. When I try do it use the tab "Compile" and the option "Build" inside Flair interface; it shows errors mentioning g77. But ok, perhaps I'm not using it right.
>
>
> Attached file: USRBIN comparing Photon Flux in the same region in a
> single-step simulation and in the 2nd step :-)
>
> Once more, thanks too much for patience, time and help.
>
> Marlon
>
>
>
> -----Mensagem original-----
> De: Francesco Cerutti [mailto:Francesco.Cerutti_at_cern.ch]
> Enviada em: quinta-feira, 10 de março de 2016 12:48
> Para: Marlon Saveri Silva <marlon.saveri_at_lnls.br>
> Cc: Mao, Stan <mao_at_SLAC.Stanford.EDU>; fluka-discuss_at_fluka.org
> Assunto: Re: RES: [fluka-discuss]: RES: Splitting FLUKA runs
>
>
> Hi Marlon,
>
> I see a main conceptual flaw here [apart from the fact that one should always ask him/herself if he/she really needs two steps].
> One (right) thing is to dump million particles in the first step and then read all of them at initialization in your second step source.f. If you properly set the dimension of your source arrays, you do not meet any problem with that.
> Another (wrong) thing is to load, as you do, ALL particles at EACH second step primary event. This way you have a first step primary generating on average, according to what you wrote, 0.47 dumped particles, and in each second step history you want to load million particles! This makes no sense and certainly breaks the stack size, that by the way you cannot alter (as reminded many times on this list, one cannot change the parameters in the included files of $FLUPRO/flukapro, since the FLUKA library has been precompiled with the distributed values). So you should rather sample ONE particle at a time from your arrays: if you have read NLINES particles, then choose ISAMPL = FLRNDM (DUMMY) * NLINES ISAMPL = ISAMPL + 1 and take the relevant properties of the particle #ISAMPL (PART(ISAMPL),X(ISAMPL), etc), removing the illogical cycle
> DO 40 MYCOUNT=1, NLINES
> NPFLKA = NPFLKA + 1
>
> Then, what matters for the second step error (without forgetting that
> one should also have a clue on the first step error) is number of
> (second
> step) primaries times number of cycles, which means total statistics (still you need at least a few cycles to let FLUKA postprocessing tools evaluate the statistical error, and let you judge it).
>
> Finally, from what you wrote, I cannot understand what is your problem
> with the compilation inside Flair
>
> Kind regards
>
> Francesco
>
> **************************************************
> Francesco Cerutti
> CERN-EN/STI
> CH-1211 Geneva 23
> Switzerland
> tel. ++41 22 7678962
> fax ++41 22 7668854
>
> On Wed, 9 Mar 2016, Marlon Saveri Silva wrote:
>
>>
>> Dear Vittorio and all,
>>
>>
>>
>> I had misunderstood a comment about remove all statements in
>> MGDRAW ENTRY. Thanks to your answer, now mgdraw.f (attached) is
>> compilling and linking!!!
>>
>>
>>
>> The next step was edit source.f and it has seemed to be right,
>> however when I compared photon fluence in the 1st step with photon
>> fluence in the 2nd step in a same section it was a little different
>> (ComparingSteps1E4.PNG).
>>
>>
>>
>> When I repeat using more primaries, difference still occurs and
>> after some hours I realized the source.f is able to read only the
>> first 40000 lines from USERDUMP output file, because parameter NLINES
>> is limited to 40000.
>>
>>
>>
>> It’s a big problem, because I need do such simulation using at
>> least 1E8 primaries (in this case, each primary generates 0.47
>> particles in that section… So I need to use a file with 4.7E6 rows).
>>
>>
>>
>> I imagined it happens because MFSTCK parameter inside the file
>> (DIMPAR) is set as 40000. Therefore, I tried change MFSTCK to
>> 50000000, but after this I couldn’t compile my new source.f anymore.
>> When I changed MFSTCK to
>> 50000 it had compiled, however Flair simulation finished with
>> TIMED.OUT
>>
>>
>>
>> It’s using gfortran (export FLUFOR=gfortran); otherwise source.f
>> was not compiling; by the way, I suppose that’s the reason I can’t
>> compile inside Flair.
>>
>> Linux version file: “Linux version 3.19.0-47-generic
>> (buildd_at_lgw01-19) (gcc version 4.8.2 (Ubuntu 4.8.2-19ubuntu1) ) […]”
>>
>>
>>
>> Finally, about second step, since each primary means NLINES
>> particles (and
>> 1 primary from 1st step); in order to have a smaller error, is it
>> better use more primaries or more cycles?
>>
>>
>>
>> =====================================================================
>> =
>> =====
>> =====================================================================
>> =
>> =====
>> ====
>>
>> Some notes (summary) about editing mgdraw.f and source.f if someone
>> intent to use in the future:
>>
>>
>>
>> 1. Use CHARACTER*8 MRGNAM, NRGNAM in order to use region names
>> instead of numbers
>>
>> 2. I typed WRITE(88,*) JTRACK,XSCO,YSCO,
>> ZSCO,CXTRCK,CYTRCK,Ecin,WTRACK in just one line, without spaces
>> because FORTRAN is limited to 72 characters by line
>>
>> 3. This way, USERDUMP generates a file with “spaces” as separator,
>> but source.f needs “tab”. Therefore it needs be converted. It can be
>> made using excel, however excel has a limited number of rows.
>>
>> 4. Likewise, USERDUMP generates an output using E as 10^ (e.g: 100 =
>> 1.0E+2); the input works with D (100 = 1.0D+2)
>>
>> 5. Besides that, if 1st step is made using more than 1 spawn, output
>> files need be concatenated
>>
>> 6. Type of particle needs be defined as INTEGER instead of DOUBLE
>> PRECISION;
>> i.e.: INTEGER PART(NLINES)
>>
>> 7. To link mgdraw.f, use Vittorio suggestion:
>>
>> $FLUPRO/flutil/fff $FLUPRO/usermvax/mgdraw.f
>>
>> $FLUPRO/flutil/lfluka -m fluka $FLUPRO/usermvax/mgdraw.o
>>
>> 8. To compile TwoStepsSource.f, use:
>>
>> cd $FLUPRO
>>
>> export FLUFOR=gfortran
>>
>> $FLUPRO/flutil/fff $FLUPRO/flutil/TwoStepsSource.f
>>
>> $FLUPRO/flutil/lfluka -o TwoStepsSource -m fluka
>> $FLUPRO/flutil/TwoStepsSource.o
>>
>> 9. It’s easier for me open .dat using OPEN Card instead of using “open”
>> inside source.f
>>
>>
>>
>>
>>
>> De: Vittorio Boccone [mailto:dr.vittorio.boccone_at_ieee.org]
>> Enviada em: sábado, 5 de março de 2016 21:12
>> Para: Marlon Saveri Silva <marlon.saveri_at_lnls.br>
>> Cc: Mao, Stan <mao_at_slac.stanford.edu>; fluka-discuss_at_fluka.org
>> Assunto: Re: [fluka-discuss]: RES: Splitting FLUKA runs
>>
>>
>>
>> Hi Marlon, seems you have heavily changed the mgdraw removing SODRAW,
>> ENDRAW etc.. therefore you custom mgdraw cannot anymore replace the default one.
>> This two lines should do the compilation
>>
>>
>>
>> :~/Software/fluka-list$ $FLUPRO/flutil/fff my_mgdraw.f
>>
>> :~/Software/fluka-list$ $FLUPRO/flutil/lfluka -m fluka my_mgdraw.o
>>
>>
>>
>>
>>
>> Plus you might have several libraries missing.
>>
>> Which Linux version and which gfortran (or g77) version are you using?
>>
>> Best,
>>
>> V.
>>
>>
>>
>> On 04 Mar 2016, at 21:08, Marlon Saveri Silva
>> <marlon.saveri_at_lnls.br> wrote:
>>
>>
>>
>> Dear FLUKA experts,
>>
>> Here I am again. :/
>>
>> This solution (making a two steps simulation) was suggested at the
>> topic "Estimating hardware configurations for bremsstrahlung
>> simulations",
>>
>> In my last e-mail, I present a big mistake trying to score a lot of
>> values (energy, type, position, etc) using a combination of USRBDX,
>> USRBIN and FORTRAN programming with a wrong Gaussian function.
>>
>> The right solution for doing this two-steps simulation is using
>> USERDUMP card according to some publications/discussions:
>>
>> https://www.fluka.org/free_download/course/triumf2012/Workshop/Laxdal.
>> pdf
>> http://www.fluka.org/web_archive/earchive/new-fluka-discuss/6806.html
>> http://www.fluka.org/web_archive/earchive/prova/0395.html
>>
>> So, I introduced USERDUMP, edited mgdraw.f (attached) and I typed "
>> $FLUPRO/flutil/fff $FLUPRO/usermvax/mgdraw.f" to create mgdraw.o.
>>
>> It works. So, the next step here is understand why I can not generate
>> the executable using any of the codes below (found on internet) and
>> understand how to read and concatenate such output files.
>> Nevertheless, the existence of this BXDRAW function left me quite
>> optimistic.
>>
>> $FLUPRO/flutil/lflukac -m fluka -C $FLUPRO/usermvax/usrini.f
>> $FLUPRO/usermvax/usrout.f $FLUPRO/usermvax/mgdraw.f -o flukaseamu
>> $FLUPRO/flutil/lfluka -m fluka -o flukamy $FLUPRO/usermvax/mgdraw.o
>> $FLUPRO/flutil/lfluka -o mgdraw $FLUPRO/usermvax/mgdraw.f
>> $FLUPRO/flutil/lfluka -o $FLUPRO/MyFluka -m fluka
>> $FLUPRO/flutil/s160302.o $FLUPRO/usermvax/mgdraw.o
>>
>>
>> Thanks again,
>> Marlon
>>
>>
>> Some errors showed:
>>
>> /opt/fluka/libflukahp.a(mgdraw.o): In function `mgdraw_':
>> /home/alfredo/fluprogfor/usermvax/mgdraw.f:6: multiple definition of
>> `mgdraw_'
>> /tmp/cceb4mg7.o:/opt/fluka/usermvax/mgdraw.f:6: first defined here
>> /opt/fluka/libflukahp.a(mgdraw.o): In function `bxdraw_':
>> /home/alfredo/fluprogfor/usermvax/mgdraw.f:105: multiple definition
>> of `bxdraw_'
>> /tmp/cceb4mg7.o:/opt/fluka/usermvax/mgdraw.f:67: first defined here
>> /usr/bin/ld: cannot find -lmathlib
>> /usr/bin/ld: cannot find -lpawlib
>> /usr/bin/ld: cannot find -lgraflib
>> /usr/bin/ld: cannot find -lgrafX11
>> /usr/bin/ld: cannot find -lpacklib
>> /usr/bin/ld: cannot find -lkernlib
>> collect2: error: ld returned 1 exit status
>> ------------------------------------------------------
>>
>> gfortran: error: -m: No such file or directory
>> gfortran: error: fluka: No such file or directory
>> gfortran: error: -o: No such file or directory
>> gfortran: error: flukamy: No such file or directory
>> ------------------------------------------------------
>>
>> collect2: error: ld returned 1 exit status
>> ------------------------------------------------------
>>
>> /opt/fluka/libflukahp.a(mgdraw.o): In function `mgdraw_':
>> /home/alfredo/fluprogfor/usermvax/mgdraw.f:6: multiple definition of
>> `mgdraw_'
>> /opt/fluka/usermvax/mgdraw.o:/opt/fluka/usermvax/mgdraw.f:6: first
>> defined here
>> /opt/fluka/libflukahp.a(mgdraw.o): In function `bxdraw_':
>> /home/alfredo/fluprogfor/usermvax/mgdraw.f:105: multiple definition
>> of `bxdraw_'
>> /opt/fluka/usermvax/mgdraw.o:/opt/fluka/usermvax/mgdraw.f:67: first
>> defined here
>> /usr/bin/ld: i386 architecture of input file
>> `/opt/fluka/flutil/s160302.o' is incompatible with i386:x86-64 output
>> /opt/fluka/flutil/s160302.o: In function `source_':
>> /opt/fluka/flutil/s160302.f:133: undefined reference to `s_wsle'
>> /opt/fluka/flutil/s160302.f:133: undefined reference to `do_lio'
>> /opt/fluka/flutil/s160302.f:133: undefined reference to `do_lio'
>> /opt/fluka/flutil/s160302.f:133: undefined reference to `e_wsle'
>> /opt/fluka/flutil/s160302.f:159: undefined reference to `s_rnge'
>> /opt/fluka/flutil/s160302.f:159: undefined reference to `s_rnge'
>> /opt/fluka/flutil/s160302.f:159: undefined reference to `s_rnge'
>> /opt/fluka/flutil/s160302.f:159: undefined reference to `s_rnge'
>> /opt/fluka/flutil/s160302.f:159: undefined reference to `s_rnge'
>> =================================================================
>>
>> USERDUMP configurations: Type Dump, What Complete, Unit 96, Score
>> Local Losses
>>
>> *...+....1....+....2....+....3....+....4....+....5....+....6....+....
>> 7
>> ....+
>> ....
>> USERDUMP 100. 96. 3.
>> Dump
>>
>>
>> -----Mensagem original-----
>> De: Mao, Stan [mailto:mao_at_slac.stanford.edu] Enviada em: sexta-feira,
>> 4 de março de 2016 15:06
>> Para: 'fluka-discuss_at_fluka.org' <fluka-discuss_at_fluka.org>
>> Cc: Marlon Saveri Silva <marlon.saveri_at_lnls.br>
>> Assunto: RE: Splitting FLUKA runs
>>
>> Please help Marlon on his question.
>> Thanks
>> Stan
>>
>> -----Original Message-----
>> From: Mao, Stan
>> Sent: Friday, March 04, 2016 10:05 AM
>> To: 'fluka-discuss_at_fluka.org'
>> Cc: 'marlon.saveri_at_lnls.br'
>> Subject: FW: Splitting FLUKA runs
>>
>>
>>
>> -----Original Message-----
>> From: Marlon Saveri Silva [mailto:marlon.saveri_at_lnls.br]
>> Sent: Friday, March 04, 2016 5:27 AM
>> To: hvincke_at_slac.stanford.edu; Mao, Stan; Rokni, Sayed H.
>> Cc: Roberto Madacki; MARILIA LISBOA DE OLIVEIRA; Beamlines Common
>> Systems
>> Subject: Splitting FLUKA runs
>>
>> Dear Mr. Vincke, Mao and Rokni,
>>
>>
>>
>> In December 2003 there was a very interesting
>> publication from SLAC (SLAC-PUB-10010 - FLUKA Calculations for the
>> Shielding Design of the SPPS Project at SLAC), in which FLUKA
>> simulation were made in order to determine shielding.
>>
>> A greatly important point is quoted in the 4th page:
>> "the calculations were split into two runs. In a first set of runs
>> (called RUN 1) information of particles reaching a virtual plane
>> outside the FFTB north wall was stored on a file" and such
>> information
>> included: "number of total primaries, total weight of the primary
>> praticles, energy, particle type, weight, co-ordinates [and]
>> direction cosines".
>>
>> I've been working on FLUKA simulations in order to
>> estimate life of some equipment subject to high dose inside a
>> beamline and, recently, aiding the Radiological Protection Group with
>> other calculations for Sirius beamlines.
>>
>> Perhaps you have seen my topic at FLUKA-discuss forum
>> some days ago about the difficulty in getting faster simulations and
>> how to relate time and hardware configuration. One of the indicated
>> solution is splitting the simulation in two or more parts as you did
>> in your work thirteen years ago.
>>
>>
>>
>> In a perfect world, there would be a card in FLUKA (RUN1) wherewith
>> we just draw a plane and it would give us a file containing all
>> information to be exported as an input file for RUN2 (it would make
>> me very happy).
>>
>> I've started trying an alternative way using USRBDX
>> card to get spectrum of photons, using USRBIN card to get the
>> position distribution in this transversal section of the pipe and,
>> finally, trying to generate a FORTRAN code inside source.f (based on
>> random
>> numbers) to create an input for RUN2 similar to those outputs of RUN1.
>>
>> However, I'm having troubles programming it and the
>> new input for RUN2 is different from RUN1 output (as you can check in
>> the attached example). Moreover, this solution don't give me
>> information about particle type, weight, direction cosines and I'm
>> not sure about the best way to select energies (random vs import from
>> a discrete table).
>>
>>
>>
>> Therefore, if you may share, I'd like to understand how have you done
>> such splitting.
>>
>>
>>
>> Thanks,
>>
>>
>>
>> Marlon Saveri Silva
>>
>> Mechanical Engineer
>> Beamlines Instrumentation and Support Group - SIL Brazilian
>> Synchrotron Light Laboratory- LNLS - CNPEM
>> + 55 (19) 3512-2490
>>
>> + 55 (19) 99478-8097
>>
>>
>>
>> <mgdraw.f>
>>
>>
>>
>>
>>
>
(Ƨځ맲r*'~&x%Zm 0nߖi'iw
Received on Tue Mar 15 2016 - 15:31:56 CET