@sblionel I was addressing this statement
It’s OK that the random sequences are the same between multiple runs of the same program.
Milan:
I am just trying to work out your code and see how I then change it to do vehicles. So I have amended the code to give me different sequences, I will use Dr Fortran’s method.
I have had a strong dislike for Fortran Random number generators for 30 years, the MS Basic Random Number generator is superior, using that I can solve an old genetic algorithm from SciAm and Fortran gets stuck in a Random number issue for the problem. I retry the same problem every few years and it has not been solved.
All random numbers are not created equal. I was awoken early this morning to explain this problem on a practical side to a gentleman in Europe, you have a data stream of 8000 reals every 8 seconds, find the 2 numbers that are not Gaussian in the stream in real time - you are using a RPi and you are using the other cores and there is one left.
Explaining it is not simple. A normal day for a Fortran programmer, most of us explain the impossible to the great unwashed masses. Heinlein put it better.
1 Like
It is interesting to look at the results not as a global accuracy level but the accuracy of each number.
The results for your validation set for each letter are shown below:
I have run it enough times and through to 10 epochs to show these results are pretty normal. If you do a statistical analysis on the low and high values, you show that there is a SIGNICIFICANT statistical difference between the set 0,1, 4 6 7 and 2 3 5 8 and 9 with a t stat > 6 in recognition, they are different.
Anyway it is interesting.
6 is surprising, but I guess it is because the top bends to the right.
just out of interest why do you set the first iteration on the net training to 3, which is why your initial accuracy of 9.9% is not an accuracy it is merely a count of how many 3’s you have, just wondering?
1 Like
Interesting results! Nice to see somebody look into it in more detail.
I don’t understand. Can you post the relevant snippet?
net = network([ input(net1), dense(net2), dense(net3) ])
call net%print_info()
write(*,130)
130 Format(/," End printing AI Model Data ",//"----------------------------------------------------------------------------------------------------",/)
!if (this_image() == 1) &
print '(a,f5.2,a)', ' Initial accuracy: ', accuracy(net, validation_images, label_digits(validation_labels),1) * 100, ' %'
So you have an initial accuracy measurement. It always comes out as about 10%. I always code in debug, and so checking the number that is popping up to be checked at this stage it is 3.
I am just going into the body to find out why, but it appears you have said on first init every thing is 3, or is it random init and 3 is just the dominant number at the start of the analysis. Either fits the pattern.
Sorry, I still don’t follow. In the MNIST example, the accuracy is calculated over the entire validation dataset (10000 samples). The accuracy there is the ratio of correctly recognized samples over the total number of samples.
program Zeus
use Base
use Draw
use nf, only: dense, input, network
use nf_datasets_mnist, only: label_digits, load_mnist
use nf_optimizers, only: sgd
implicit none
type(network) :: net
real, allocatable :: training_images(:,:), training_labels(:)
real, allocatable :: validation_images(:,:), validation_labels(:)
integer :: n, num_epochs, nframes ! nframes is the number of pictures in the data set.
integer :: typ ! File name
integer :: counter(mn_10,mn)
real ave,adev,sdev,var,skew,curt, answer
num_epochs = 10
print '(//" MNIST DATA ANALSIS USING AI",/)'
print '(80("-"),//)'
write(*,120)num_epochs
120 Format(" Number of epochs :: ", i6,//,"----------------------------------------------------------------------------------------------------",//," Printing the AI Model Format.")
write(*,100)
100 Format(/" Loading Training Data Files")
open(st,file = "Results.log",status = "UNKNOWN")
call load_mnist(training_images, training_labels, validation_images, validation_labels)
nframes = 10
typ = 1
call TrainWrite(training_images, num_training_images, image_size, nframes, typ)
call TrainStats(training_labels, validation_labels, num_training_images, num_validation_images,counter)
call moment(training_labels, num_training_images, ave, adev, sdev, var, skew, curt)
call moment2(counter, mn_10, mn, ave,adev,sdev,var,skew,curt,1)
call moment2(counter, mn_10, mn, ave,adev,sdev,var,skew,curt,2)
nframes = 2
typ = 2
call TrainWrite(validation_images, num_validation_images, image_size, nframes, typ)
write(*,110)net1,net2,net3
110 Format(/" Loading Network",//" Input Vector Length :: ",i6,/" Dense Vector Length :: ",i6,/," Second Dense Vector Length :: ",i6 )
net = network([ input(net1), dense(net2), dense(net3) ])
call net%print_info()
write(*,130)
130 Format(/," End printing AI Model Data ",//"----------------------------------------------------------------------------------------------------",/)
!if (this_image() == 1) &
print '(a,f5.2,a)', ' Initial accuracy: ', accuracy(net, validation_images, label_digits(validation_labels),1) * 100, ' %'
you create the network at net = network …
It has not been trained, it is just initialized.
then printout the net information.
then you run an accuracy analysis. This gives always an answer at 10%, and so this is in theory exactly the same as assigning random guesses to the answer.
So doing accuracy before training is not a step anyone needs, we know the answer will be 10%. I would drop it, but I was just wondering why you added it, your other code is terse and no waste, to the point of being good LISP.
Next question, why for the second layer did you pick 30, I am trying 50 at the moment, and did you try and add another dense layer?
I understand now. The only point of showing the accuracy before training is to confirm that it’s indeed what we expect it to be, ~10%.
I picked 30 as a happy medium between something that reaches not too terrible accuracy (~95% after 10 epochs) and that doesn’t take long on a 2014 low-end laptop. Larger hidden layer tends to be better for accuracy. Too many hidden layers may make the network unable to generalize, so I read.
The best accuracy I can get is 97% trying a few different layer sizes and adding a layer. So now I have 784, 50, 30 and 10 output.
It takes about 30 epochs to get to a peak and then it just blips about.
Have you given any thought to how the model is saved so it can be reused with out the need to retrain?
I will put in a drawer so the pictures that do not match can be viewed.
The problem is still no 5 and 9. These add most of the error.
You are entitled to your likes and dislikes, but what is the basis for your claim that the MS Basic RNG is superior? That RNG is documented at Microsoft KB Archive/231847 - BetaArchive Wiki . I ran it through L’Ecuyer’s SmallCrush battery, and the MS RNG failed every one of the tests. There have been some critical evaluations made on this RNG. A visual demonstration of its shortcomings and a suggested fix is available.
There is no universal “Fortran Random number generator”. Each Fortran compiler comes with its own RNG, and the quality of such RNGs is, therefore, variable. Nothing prevents you from coding the MS Basic RND in Fortran or your “Fortran RNG” in Basic.
1 Like
Scientific American used to publish a Recreational Mathematics column, written by Gardner and others. If I remember correctly in the 1980s someone other than Gardner published a genetic algorithm. I had access to a PC computer and MS Basic. It took me a few days to get the program coded and running and it is just plain fun to play with, then and now.
When I bought MS Fortran, I tried to get the algorithm running in Fortran. The algorithm runs fine, but you need a RNG to get the code to work properly. The program converges to an answer, BASIC would land on the answer quickly and Fortran would just loop and loop.
I tried Soup again recently on Modern Fortran and I found a Basic program to run on Windows 10. I could not see a change in the results.
I am probably missing something in the MS Fortran RNG, but blowed if I can see what it is. The SOUP program is stored on a post in Intel Fortran Forum.
If you just need run of the mill random numbers it is fine, but all random numbers are not created equal. Every RNG has to interact with the system and this makes them not truly random. A way to check it is to take say 10 million numbers, do an FFT and look for the peaks. If there are any large ones then that is system or algorithm flaw. Just as Gaussian random thermal after vibrating a beam is no longer Gaussian at all frequencies.
The genetic algorithm looked interesting at the time.
I would not write a RNG, find some truly random generating event and use that to generate an endless string.
So in essence the stream from Fortran RNG most likely repeats in some form and then soup cannot break out of the repeat, is my honest best guess.
I am not critiquing MS, these things are deuced hard. Plus I do not go looking for more work.
Earlier versions have savetxt and loadtxt methods on the network instance to save and load to and from a simple and ad hoc text file. The last version that has this is 0.2.0.
With the recent refactor I haven’t bothered porting savetxt and loadtxt for two reasons:
- HDF5 is the way to go to allow reading Keras and PyTorch models. I’m being paid to implement this for Keras.
- Users that are saving and loading models are mostly doing it via FKB which uses neural-fortran under the hood for inferrence, but uses text files to load Python models in Fortran applications.
From your snippets, it looks like you are working with the recent refactor (great!). If you don’t want to wait for HDF5 you can just loop over layers and for each layer write its weights and biases arrays into a text file (list-directed I/O is fine). And similar for loading models from file: Instantiate a new network, read the weights and biases from file, and assign the values to the layer components.
Thanks, that is what I thought, but I wanted to check first.
I have a small program to capture bitmaps from a webcam, so I am going to try and get it to recognize cars and trucks etc. But first I need a lot of pictures.
I looked at his Soup program, which was posted on the Intel Forum. What he calls the “Fortran RNG” is his own random number generator, which he coded in Fortran, and has nothing to do with the various library routines provided by the MS Fortran compiler for obtaining random numbers.
Macneacail’s “Fortran RNGs” take single digits from the values returned by the MS Fortran GETTIM subroutine, and twiddle and mix those digits to obtain his random numbers. I suspect that this is a poor way to construct an RNG.
It is inappropriate to label the unspecified “Fortran RNG” as being poor in quality. He did not use any of the Fortran-supplied RNGs. Instead, he wrote his own, and that turned out to be inadequate. That is all. A poor RNG algorithm is poor no matter in which programming language it was coded.
I created a separate post on Soup. I have tried many RNG’s on Soup. The posted one is just a play toy trying out ideas. The Fortran RNG has never achieved 100 in 35 years, the Basic program RNG gets there every time.
I suggest you try to solve the problem, it was beyond my skill set.