Is this a missing file or just another Apple mex issue?

[zmichels]

I just downloaded the 5.11 mtex (https://github.com/mtex-toolbox/mtex/releases/download/mtex-5.11.1/mtex-5.11.1.zip) and went to install on my Apple laptop. I am familiar with a number of install issues and workarounds regarding the mex files, their compilation, and installation. When I run this latest startup_mtex, all of the present mex files compile, but I get the following error about a missing file:

Error using check_installation>fast_check_mex
missing mex-file jcvoronoi_mex.mexmaci64

Is this in-fact an error due to a file that is missing from the install folder? Or is it just some new artifact of the Apple+mex+MTEX installation nightmare?

Thanks for any info or thoughts.

Cheers,
Zach

[lfgmorales]

Hi Zach You can try downloading this <4384458> [4384458.png] jcvoronoi binary for ARM-Mac · ***@***.***<4384458> github.com<4384458> and put the file on mtex—>extern—>jcvoronoi OR inside the mex folder, although I believe it ahould be there from your installation Cheers Luiz

[kilir]

Maci is for intel macs, the arm binary doesn'do anything there.

[kilir]

Hi Zach,

if you run mex_install (or mex('-R2018' ....) in extern/jcvoronoi you might be able to build it for intel mac. However, I think currently the build most likely fails using clang due to having -Werror enabled by default in macos, which escalates all warnings to errors (and there's a type casting warning not yet fixed).

For now, either you could try to find a way to pass -Wno-error as a compiler flag and build a nice intel mac binary (which could then also be distributed), or find a way to use a real gcc (https://github.com/danfortunato/matlab-gcc) which seems also to be more forgiving with the warning, or maybe the person who build the arm macos binary could help out since the same issue persited there as well.

An unsatisfying workaround would be to use a different voronoi method (see mtex_settings), however this way you wouldn't benefit from the up to 20x speedup in grain computation.

Cheers,
Rüdiger

[zmichels]

Yeah. I thought it was rough getting students setup before, but now none of my students (or myself) can get 5.11.x working (no configuration of Matlab versions + alternative compilers + different mex files are working for us), so we are planning to just keep using 5.10.x (at least we've got a system of functional workarounds for installing 5.10.x versions) until something readily installable comes along.

[kilir]

Adam,
you are building an old version of jcvoronoi. Is there any reason you use the old status at cf3d00b ?

This one should problems, also with clang.
https://github.com/mtex-toolbox/mtex/blob/develop/extern/jcvoronoi/jcvoronoi_mex.cpp

[kilir]

Zach,
so you are all on arm macOS, right?

https://github.com/mtex-toolbox/mtex/tree/develop/mex
includes the mexmaca64 and those do not work for you?

Cheers,
Rüdiger

[zmichels]

Rüdiger,

That's correct. I'm not claiming it's undoable by any means, but we haven't sorted out how to get it working. I'm not crushed by this as we are getting along fine with 5.10.x for now. It's only 3 students, and they're doing fairly routine stuff. I mainly wanted it for the faster grain construction... but usually I just do that once and save the data, so it's no problem.

[creuzige]

Adam,
you are building an old version of jcvoronoi. Is there any reason you use the old status at cf3d00b ?

@kilir I think I entered the date wrong initially, unless 2 days ago is really the 'old' status. It also sounds like @ralfHielscher there's a 5.11.2 release coming out soonish. Happy to beta test if you'd like.

Thanks!

[zmichels]

Hi all,

Thanks for the suggestions!

Update:

I used a fresh homebrew install of gcc and set it to be used in Matlab (as mentioned in the link provided by Rüdiger). Using this setup, I am able to "successfully" complete the MTEX startup for 5.11.1.

Even after I get the jcvoronoi_mex file compiled with gcc and MTEX 5.11.1 starts without errors, any attempt to construct grains gives an error that the mex file (jcvoronoi_mex.mexmaci64) is invalid and throws errors about "missing" files (i.e., libMatlabEngine.dylib) that are not missing (confirmed they are present in the indicated folder)... and it all seems [from some limited internet searching] perhaps related to the scattered and mismatched state of support in Matlab for the latest Apple silicon. But I am speculating and don't know what I am talking about.

In the end, I achieved an error-free-startup MTEX 5.11.1 fairly easily, but the installation did not work properly. Not sure what to modify for a better result, but I'm open to any suggestions.

[kilir]

Hi Zach,

just to be sure. You are using an intel mac, not an arm mac with a an intel matlab, right?

Cheers,

Rüdiger

[zmichels]

I have an ARM Mac for the last 6 months — and I have been using 2023b without trouble (for intel I believe?). I have tried to install MTEX 5.11.1 on both the 2023b and 2024a (for ARM but with Rosetta?), and had no luck with either. For now, I’m continuing on 2023b because 5.10.x runs great on it, but I cannot get either to build correctly for 2024a.Cheers,ZachOn Apr 2, 2024, at 12:43 AM, Rüdiger Kilian ***@***.***> wrote: Hi Zach, just to be sure. You are using an intel mac, not an arm mac with a an intel matlab, right? Cheers, Rüdiger —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***>

[kilir]

Ah ok, sorry I was assuming that you're on intel since the maci matlab was looking for is an intel binary.
In that case, if you'd use the native arm matlab (no rosetta) mtex should actually not search for maci but maca instead and the mex file provided seems to work. any chance to test if the maci would still be requested in an arm-only environment?

Cheers,
Rüdige

[Tijmenvermeij]

Sorry, I mistakingly put some ChatGPT prompts here... 😅 deleted

[lisakemp]

Hi, MTEX Toolbox group – I’m Lisa Kempler from MathWorks. @tijmen – If you believe your error may be related your MEX-file (.mex) implementation and your debugging efforts aren’t getting you to a resolution, consider contacting MathWorks technical support: ***@***.******@***.***>. I recognize this is C++ code, but worth a try, especially if you believe the problem maybe be related to the MATLAB to C++ interface. @zachary Michels - Is this related to your original issue with this? Is that resolved? I’ll look for a post from you if it gets resolved in the meantime. Lisa From: Tijmen Vermeij ***@***.***> Sent: Wednesday, April 17, 2024 6:11 AM To: mtex-toolbox/mtex ***@***.***> Cc: Subscribed ***@***.***> Subject: Re: [mtex-toolbox/mtex] Is this a missing file or just another Apple mex issue? (Discussion #2106) I'm going to put the c++ code and the include file below, please check if there any potential error sources, especially related to "type casting":

[lisakemp]

Did you report this to MathWorks technical support? ***@***.*** and @adam Creuziger) From: Zachary Michels ***@***.***> Sent: Tuesday, April 23, 2024 11:19 AM To: mtex-toolbox/mtex ***@***.***> Cc: Lisa Kempler ***@***.***>; Comment ***@***.***> Subject: Re: [mtex-toolbox/mtex] Is this a missing file or just another Apple mex issue? (Discussion #2106) Yeah. I thought it was rough getting students setup before, but now none of my students (or myself) can get 5.11.x working (no configuration of Matlab versions + alternative compilers + different mex files are working for us), so we are planning to just keep using 5.10.x (at least we've got a system of functional workarounds for installing 5.10.x versions) until something readily installable comes along. — Reply to this email directly, view it on GitHub<#2106 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ADCNSXNHDPII24KZYXLN6RDY6Z3UNAVCNFSM6AAAAABFQDO2ECVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TEMBSGM4DS>. You are receiving this because you commented.Message ID: ***@***.***>

[ralfHielscher]

Hi Zach, I'm currently working on MTEX 5.11.2 which will ship with new Mac binaries and more checks. I will let you know when it is ready. Ralf.

…

On Tue, 23 Apr 2024 at 18:23, Zachary Michels ***@***.***> wrote: Rüdiger, That's correct. I'm not claiming it's undoable by any means, but we haven't sorted out how to get it working. I'm not crushed by this as we are getting along fine with 5.10.x for now. It's only 3 students, and they're doing fairly routine stuff. I mainly wanted it for the faster grain construction... but usually I just do that once and save the data, so it's no problem. — Reply to this email directly, view it on GitHub <#2106 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACBYJ2MH2BJ2UFHRJNYO5C3Y62DG5AVCNFSM6AAAAABFQDO2ECVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TEMBTGE3DG> . You are receiving this because you commented.Message ID: ***@***.***>

[zmichels]

Thanks @ralfHielscher and @kilir for everything you do.
You all are saints.

[ralfHielscher]

MTEX 5.11.2 is out and ready for testing

https://github.com/mtex-toolbox/mtex/releases/download/mtex-5.11.2/mtex-5.11.2.zip

There is no automatic check at startup. You may want to type

check_mex

to get a detailed overview about which mex files are working and which not.
You can do

mex_install force

to recompile the mex-files on your own computer.

Any feedback is appreciated.
Ralf.

[creuzige]

@ralfHielscher

Install went ok, but trying to run a script that worked under 5.11.1 failed.

check_mex output:

>> check_mex

Mex files are compiled binaries that are used within MTEX to speed up time critical computations. As mex files are specific to different 
operating systems it is not so easy for MTEX provide working binaries for different types of systems.

I'm now going to check all the mex files in

  /Users/creuzige/Documents/MATLAB/Mtex_versions/mtex-5.11.2/mex

In case some of the mex files are not working, you have two options
 1. Use the command mex_install to compile the mex files yourself
    On a Mac this requires that you install XCode first.
 2. Switch to a slower Matlab based implementation.

 checking: jcvoronoi_mex.mexmaci64 ok
 checking: EulerCyclesC.mexmaci64  downloading data from 
 
   https://raw.githubusercontent.com/mtex-toolbox/mtex/develop/data/EBSD/Forsterite.ctf
 
  and saving it to /Users/creuzige/Documents/MATLAB/Mtex_versions/mtex-5.11.2/data/EBSD/Forsterite.ctf
 saving data to /Users/creuzige/Documents/MATLAB/Mtex_versions/mtex-5.11.2/data/small.mat
 ok
 checking: insidepoly_dblengine.mexmaci64 ok
 checking: wignerTrafo.mexmaci64 saving data to /Users/creuzige/Documents/MATLAB/Mtex_versions/mtex-5.11.2/data/dubna.mat
  0 |  0.91  0.93  0.74  0.80  0.93  0.85  0.77
  1 |  0.90  0.89  0.68  0.63  0.93  0.85  0.71
  2 |  0.87  0.86  0.56  0.59  0.90  0.81  0.66
  3 |  0.81  0.79  0.46  0.49  0.87  0.77  0.60
  4 |  0.73  0.71  0.39  0.45  0.82  0.74  0.55
  5 |  0.68  0.66  0.37  0.43  0.80  0.72  0.53
  6 |  0.67  0.63  0.35  0.42  0.79  0.71  0.52
 saving data to /Users/creuzige/Documents/MATLAB/Mtex_versions/mtex-5.11.2/data/dubnaodf.mat
 ok
 checking: wignerTrafoAdjoint.mexmaci64 ok
 checking: numericalSaddlepointWithDerivatives.mexmaci64 ok
 checking: S1Grid_find.mexmaci64 ok
 checking: S1Grid_find_region.mexmaci64 ok
 checking: S2Grid_find.mexmaci64 ok
 checking: S2Grid_find_region.mexmaci64 ok
 checking: SO3Grid_dist_region.mexmaci64 ok
 checking: SO3Grid_find.mexmaci64 ok
 checking: SO3Grid_find_region.mexmaci64 ok
 checking: nfftmex.mexmaci64 ok
 checking: fptmex.mexmaci64 ok
 checking: nfsftmex.mexmaci64 ok
 checking: nfsoftmex.mexmaci64 ok

check complete

I ran into the following error running a script. I think there's something up with the h5oina interface:

Dot indexing is not supported for variables of this type.

Error in loadEBSD_h5oina (line 302)
        rpts(n) = strcmpi(CS{namePairs(n,1)}.mineral,CS{namePairs(n,2)}.mineral);

Error in EBSD.load (line 86)
  ebsd{k} = feval(['loadEBSD_',char(interface)],...

Error in E240424_AAC_712 (line 50)
ebsd = EBSD.load(fname,CS,'interface','h5oina');

When I try to load the file from the import wizard, I get a 'Could not detect file format' error...

[ralfHielscher]

Hi Adam,

I can see where this bug was introduced. For the moment you can simply copy and paste the file loadEBSD_h5oina.m from MTEX 5.11.1 into the new version. And would also be nice if you could send me the corresponding EBSD data file such that I can fix the bug.

Ralf.

[creuzige]

Copying the file works! The data file I was working on is about 100 Mb. Would it be better if I found a smaller .h5oina file?

PS - on my data set here was the speedup for running calcGrains
v5.11.1 - setMTEXpref('voronoiMethod','qhull');
Elapsed time is 32.626104 seconds.

v5.12.2 - setMTEXpref('voronoiMethod','jcvoronoi');
Elapsed time is 13.099079 seconds.

% using https://mtex-toolbox.github.io/MaParentGrainReconstruction.html

tic

% grain reconstruction
[grains,ebsd.grainId] = calcGrains(ebsd('indexed'), 'angle', 3*degree);

% remove small grains
ebsd(grains(grains.grainSize < 3)) = [];

% reidentify grains with small grains removed:
[grains,ebsd.grainId] = calcGrains(ebsd('indexed'),'angle',3*degree);
grains = smooth(grains,5);

toc

[ralfHielscher]

If you could share it via dropbox or similar it is ok.

[creuzige]

Unfortunately Dropbox is disallowed for me, but you should have gotten an email message with a link to the data. If that doesn't work out, I can look into other options...

[zmichels]

Well, it installed without error for me, which is great! Thank you!
However, I get very different results in my standard analyses than I have gotten in previous versions of MTEX since 4.2.x.
I submitted a new issue about it to separate from this thread.