My contention all along re MoA and array languages.
Odd, first of all the reference to classic Fortran programs - things have moved on quite a bit. Second, while I admit not having watched the talk, the abstract does not give a hint to any alternatives and it also bypasses the fact that tensors as used in mathematics and physics are very much multi-dimensional arrays. Thirdly, what is unstructured about them?
Okay, this is probably preaching to my own parish, so I will stop here.
1 Like
You can go through the slides:
and it seems his main point is that a parallel scan is complicated and not necessarily an array algorithm. Not sure about scan, but he also mentions FFT. That I have a quite a bit of experience of implementing in Fortran. It’s definitely an array algorithm, and Fortran delivers A+ performance on that. It’s true that you can decompose it to parts and implement it in a couple different ways, but the FFTPACK style also works really well.
My own view is that multidimensional arrays provide a tool that you can use to implement a wide range of algorithms, and once you get used to doing that, almost everything in numerical computing can be done using arrays and Fortran is an excellent tool for that. It delivers very good competitive performance.
I concur. In what I call “dimension lifting “ we take the MoA reduction (first DNF) of any nd array expression, than “lift” by partitioning shapes to provide “ intentional information” to compilers your o map to various hardware components. This is all formal, and can allow full mathematical prowess desired. I alway view the architecture as a multidimensional array so the algorithm maps to arrays using MoA.
1 Like
Yeah! I believe this 100% I also have extensive experience with the FFT : sequential parallel shared,parallel shared/distributed(all CPUs), shared(GPU) all explained in
See reduction and scan definitions in my dissertation, noting what I mentioned about sequential versus logn implementations.
Yes, yes, once one learns to “think” in arrays, Fortran brings the “thoughts” to life. For me I see everything as an array, the algorithm, it’s optimal representation in the abstract, then the optimal implemntation in the reality by mapping arrays to arrays(the data to the hardware with memory access patterns, arithmetic and communication). Love it! parallel reduce and scan are parallel algorithms in that, based on the scalar operation in reduce or scan, can be done in log n time versus sequential.
1 Like
Hi All, (More thinking and sharing)
I was just thinking about Tensors, Tensorflow, TPUs … and all are defined using matrix operations. Most of it is matrix multiplication, kronecker products, Khatri-Raos and Tracey Singh(all defined using the MoA inner and outer product (one algorithm and one circuit Realization of the Kronecker Product in VHDL using Multi-Dimensional Arrays | IEEE Conference Publication | IEEE Xplore and
Hardware Considerations for Tensor Implementation and Analysis Using the Field Programmable Gate Array | Grout, Ian; Mullin, Lenore | download
That said, what keeps us all from designing and developing something like Tensorflow in Fortran with MoA optimizations that could build upon what @Arjen @jeremie.vandenplas @wyphan and @gak are working with me on (take, drop and catenate for n-d arrays, primary axis)
We’ll have our bimonthly meeting this Friday at 2:30. Topics: test and evaluate nd take drop catenate separately and discuss testing to put them together, status of experiments with Fortran built ins and any other relevant topics to move forward –
bimonthly MoA meeting starting in 10 minutes. Topic, progress with n-d take, drop, catenate composition. All Welcome.Lenore Mullin is inviting you to a scheduled Zoom meeting.
Topic: MoA Lectures
Time: Nov 19, 2021 02:30 PM Eastern Time (US and Canada)
Every 2 weeks on Fri, until Dec 31, 2021, 4 occurrence(s)
Nov 19, 2021 02:30 PM
Dec 3, 2021 02:30 PM
Dec 17, 2021 02:30 PM
Dec 31, 2021 02:30 PM
Please download and import the following iCalendar (.ics) files to your calendar system.
Weekly: https://albany.zoom.us/meeting/tJwkfumuqT8sGdVCPn9GavN8j88zWjS399rF/ics?icsToken=98tyKuCgpzIqHNORthqGRow-HYr4c-7wtildjadrvy_rWgdSdC2uPLoaKIV1I4uJ
Join Zoom Meeting
Meeting ID: 989 9067 4631
Passcode: 122039
One tap mobile
+13126266799,98990674631# US (Chicago)
+16465588656,98990674631# US (New York)
Dial by your location
+1 312 626 6799 US (Chicago)
+1 646 558 8656 US (New York)
+1 301 715 8592 US (Washington DC)
+1 346 248 7799 US (Houston)
+1 669 900 6833 US (San Jose)
+1 253 215 8782 US (Tacoma)
Meeting ID: 989 9067 4631
Find your local number: Zoom International Dial-in Numbers - Zoom
Join by SIP
98990674631@zoomcrc.com
Join by H.323
162.255.37.11 (US West)
162.255.36.11 (US East)
221.122.88.195 (China)
115.114.131.7 (India Mumbai)
115.114.115.7 (India Hyderabad)
213.19.144.110 (Amsterdam Netherlands)
213.244.140.110 (Germany)
103.122.166.55 (Australia Sydney)
103.122.167.55 (Australia Melbourne)
209.9.211.110 (Hong Kong SAR)
149.137.40.110 (Singapore)
64.211.144.160 (Brazil)
149.137.68.253 (Mexico)
69.174.57.160 (Canada Toronto)
65.39.152.160 (Canada Vancouver)
207.226.132.110 (Japan Tokyo)
149.137.24.110 (Japan Osaka)
Meeting ID: 989 9067 4631
Passcode: 122039
Join by Skype for Business
All Welcome.Lenore Mullin is inviting you to a scheduled Zoom meeting.
Topic: MoA Lectures
Time: Dec 3, 2021 02:30 PM Eastern Time (US and Canada)
Every 2 weeks on Fri, until Dec 31, 2021, 3 occurrence(s)
Dec 3, 2021 02:30 PM
Dec 17, 2021 02:30 PM
Dec 31, 2021 02:30 PM
Please download and import the following iCalendar (.ics) files to your calendar system.
Weekly: https://albany.zoom.us/meeting/tJwkfumuqT8sGdVCPn9GavN8j88zWjS399rF/ics?icsToken=98tyKuCgpzIqHNORthqGRow-HYr4c-7wtildjadrvy_rWgdSdC2uPLoaKIV1I4uJ
Join Zoom Meeting
Meeting ID: 989 9067 4631
Passcode: 122039
One tap mobile
+13126266799,98990674631# US (Chicago)
+16465588656,98990674631# US (New York)
Dial by your location
+1 312 626 6799 US (Chicago)
+1 646 558 8656 US (New York)
+1 301 715 8592 US (Washington DC)
+1 346 248 7799 US (Houston)
+1 669 900 6833 US (San Jose)
+1 253 215 8782 US (Tacoma)
Meeting ID: 989 9067 4631
Find your local number: Zoom International Dial-in Numbers - Zoom
Join by SIP
98990674631@zoomcrc.com
Join by H.323
162.255.37.11 (US West)
162.255.36.11 (US East)
221.122.88.195 (China)
115.114.131.7 (India Mumbai)
115.114.115.7 (India Hyderabad)
213.19.144.110 (Amsterdam Netherlands)
213.244.140.110 (Germany)
103.122.166.55 (Australia Sydney)
103.122.167.55 (Australia Melbourne)
209.9.211.110 (Hong Kong SAR)
149.137.40.110 (Singapore)
64.211.144.160 (Brazil)
149.137.68.253 (Mexico)
69.174.57.160 (Canada Toronto)
65.39.152.160 (Canada Vancouver)
207.226.132.110 (Japan Tokyo)
149.137.24.110 (Japan Osaka)
Meeting ID: 989 9067 4631
Passcode: 122039
Join by Skype for Business