ac7 style file renamer: testers wanted

[shiihs]

Here's a new version that should get rid of the "weird" characters tacked onto the current display name,

and that automatically adds the .AC7 extension if not typed by the user.

 

Linux executable: https://drive.google.com/file/d/11Wnep3BsOosU4xxxWKFppQCTW8S8jH26/view?usp=sharing

Windows executable: https://drive.google.com/file/d/1qsQc7rwRjGRbR8bIGe4PMdDyke0TUqrz/view?usp=sharing

 

 

[shiihs]

hmm i just noticed I made a mistake... I tested only a subset of the styles I downloaded from the forum and of course as Murphy's law will have it, the remaining styles reveal some new problems. Still not ready for prime-time...

 

[Jokeyman123]

One change I've just noticed with this latest update-now when I load a file, I see what the name would look like when I call it up on the PX560, which makes sense-as normally rhythm files I load usually end up with names that look nothing like the name listed in my thumb drive on my computer. I did not notice this on earlier versions. When I loaded a file from my drive-the name looked the same in the earlier versions of your software as it did on my thumb drive, but maybe I wasn't paying attention-this version seems to be displaying the rather cryptic name I would see once its in my PX560, and the reason I'd want to change it in the first place. So that alone is an improvement. I understand this is probably due to the DOS 8-character limitation for names inherent in the earlier Casios and in the Casio software (IDES 4.0 etc.) but its still nice to see what the old name would look like if I didn't change it. I did not notice this in the earlier versions. and Chandler, I don't think I get a 3rd and 4th variation in fills with the PX560/350 but I'll check.  And I still have the CTK6200, I'll check that too and post back. 

[shiihs]

1 hour ago, Jokeyman123 said:

One change I've just noticed with this latest update-now when I load a file, I see what the name would look like when I call it up on the PX560, which makes sense-as normally rhythm files I load usually end up with names that look nothing like the name listed in my thumb drive on my computer. I did not notice this on earlier versions. 

 

Changing the name as it is shown after importing it on the keyboard is what the tool is all about, so I'm glad it starts to work for you

In the mean time, here's a new version with the previous bugs fixed, and a proof-of-concept "rename multiple files" tab.

 

Linux executable: https://drive.google.com/file/d/1BXBYV5lSTB9lEX2cT16oA7erw2GlXToQ/view?usp=sharing

Windows executable: https://drive.google.com/file/d/16hy4FaH7LbUqqCCiHaX8FFwJOVaIGNQ6/view?usp=sharing

 

[shiihs]

(also interesting to note that you can resize the tool, so it's easier to see all the information)

 

[Jokeyman123]

I like the resizing option and the new window for batch conversions-excellent. Chandler-after checking the PX350 and older 575-there are only 2 fill-ins, regardless of which direction I go from variation 1-2 or 2-1. I'm guessing my ctk6200 and PX560 are the same.  Will check over file name length for all and post here asap. PX560 only allows for 8 characters, I'm pretty sure the 575 is the same. Will have to review the PX350. Again with this tiny screen, it's got to be only 8 character max, maybe not even. I don't use the 350 much for user programmable rhythms since it has such limited memory for that. 

 

When I created my original .ckf rhythms I posted here, I did the same as you-I created loop points so fills and variations 3-4 would be the same as 1 and 2 since i beleive all the earlier Casios only allowed for that.

 

I did not check the undocumented function for transitioning variations and fills with registrations. will try later tonight or whenever I can get to it. 

[shiihs]

New version which also allows swapping elements.

I have to admit testing on my side was a bit shallow, but what I tried worked for me...

Be sure to backup your work before you experiment!

 

Linux: https://drive.google.com/file/d/15NI3ZhGT9tVcFsym8eHOyC4jG0wG1f9E/view?usp=sharing

Windows: https://drive.google.com/file/d/17sNkGz5rCSaIO-FXCSIIIehfnHeb-AbF/view?usp=sharing

 

[Jokeyman123]

Nice update. Shuffling rhythm selection order, will have to test this out-makes it much easier to re-order everything, nice. The manual way requires using a midi editor with the Casio converter for ckf or ac7 files to change sections of the rhythm with various marker positions, but then you probably knew that already to create this.

 

If I upload and older ckf that I have converted to an .ac7-not hard, I just load a ckf into the PX560 and save it as an .ac7, would you be able to do anything with that? I have just installed Python 32-bit version 3.7, just to see what you are up to here! I have worked off command lines before, just not Python. Object-oriented, similar to Visual Basic in concept, syntax I'm sure is way different, looks closer to the Linux command syntax and programming. I'll keep checking in as long as you are asking for beta testers, nice work.

[shiihs]

3 minutes ago, Jokeyman123 said:

If I upload an older ckf that I have converted to an .ac7-not hard, I just load a ckf into the PX560 and save it as an .ac7, would you be able to do anything with that?

 

ReStyle at the moment only understands valid .ac7 files.  It shouldn't matter if they were created from .ckf or .mid or .sty or something else.
But ReStyle has no clue how to go back to .ckf - it can only write .ac7. So I think the question is if you can do something with that?

 

I was wondering about the following: do you think the element reordering controls should be hidden by default,

and shown only if the user presses some "Advanced >>>" button so as to keep the UI simpler by default?

Or is the current complexity still acceptable?

 

If you want to run from source code, I'd advise to follow all the instructions in https://github.com/shimpe/ac7renamer and also to use a decent python environment (I myself prefer the community edition of pycharm, which is free of charge, very powerful and user friendly).

 

[Jokeyman123]

IMO-keep the tabs out front-the interface so far is very uncluttered-and will show new users that this option is right there. I would think if you incorporate more features than this, might start thinking about tabs/buttons but right now I don't think its necessary. Renamer is useful for me-since only my older Casio still uses the ckf format for rhythms-as does the older IDES 4.0 from Casio. The only reason I develop custom rhythms in ckf is that many Casios will recognize that, newer and old and will convert (I know I'm repeating myself) to ac7 in the keyboard when imported and saved.  Pus there is no external editor for ac7 format, all ac7s must be customized as a user rhythm in the keyboards that can do that-even the PX560 cannot edit and save a custom rhythm internally-except for muting specific midi channels in the rhythm.  I think the MZ-X series can. so your software is a definite plus.  I'll look at pycharm when I've got a little more time. Off I go....

[shiihs]

I think the ordering of the elements in my previous version was wrong and I've now tried to make a more educated guess at the correct ordering (especially var 3, fill-in 3, var 4, fill-in 4)

It would be interesting of someone with access to a 4 variation machine could check if it's correct. Unfortunately I have no access to such keyboard.

 

Linux: https://drive.google.com/file/d/1gosUhY9-dvldb35IH93vv5rG-A2cheZd/view?usp=sharing

Windows: https://drive.google.com/file/d/1zDKr6UnoOAFzaTa6nNCpqs63xs_Uh31z/view?usp=sharing

 

 

[Chandler Holloway]

I no longer own a CT-X3000 so unfortunately I’m not much help here. I know @Brad Saucier has an MZ-X.

[Jokeyman123]

Question for shiihs-not related to this particular software, and just thinking out loud about another software development idea.....depending on your response, may have to create a separate topic or none.

 

Some Casio keyboards allow for tone, dsp and other features to be externally edited with their respective software "front-ends" such as the IDES series of editors. However, the newer CTK/WK IDES 6.0 series only allows for file transfers but is not a true software editor, although the older IDES 4.0 not only allows for full tone and DSP editing but also has a wavfile editor/manager-is a full software front-end for specific older Casios and is definitely a more useful program but is only compatible with the older Casios such as the WK3300, 3700, my PX575 and a few others. Some of the Privias have no editors at all, including my older PX350 and the newest 360/560, the latter having editing functions in the keyboard and with the large viewing screens Casio felt a software front-end would nto be needed. My question is this-I and possibly others would be willing to pay for some type of software editor for the Casios that have none.........

 

Do you think it is at all possible with your programming skills to develop some type of software editor for any of the Casios that do not have this? This would include specifically the CTK6000, 6200, 6250-7000, 7200 and the WK6500, 6600, 7500 and 7600 which probably could do with one software prgram for all of these with some slight variations-the drawbar functions that are not a part of the 6200, 6500 etc. The only older Privia I own that has no such software is the PX350 but since it cannot store any edited info that I have been able to alter externally, probably not possible to develop anything for it and other similar Privias-the 160s etc. I haven't checked but I think the newest CTX and MZ-X line has this same lack of software editor, Chandler correct me if I'm wrong although with the MZ-X large screen probably not so needed.

 

 I'm sure would be a much more difficult project but with what I 've observed with your Python skills (flattery intended!) might be doable which is why I'm asking your opinion. Might not seem like such an important development project at first glance-but again, the newest CTK and WK series have pretty deep functions built-in-are fairly daunting to work with without some kind of external software editing capabilities. IMO there are probably many CTK-WK owners still playing these, this would only add to the already very powerful abilities some of these Casios have and that many musicians using software would find very beneficial to workflow when creating music. Many Casios with the song.midi recorders built-in allow for saving multi-track work as midi file for software editing, so I don't think of that as being a priority for software development.

 

Take a look at IDES 4.0 if you need to get a rough idea of what Casio was capable of in software development as well as the PX5s and XW-P1 editors which are very nice, very useful. For computer platforms, Windows would be my first choice or whatever platform is easiest for your development. I don't have MACs but I have Windows and Linux OS on my machines, I'd be willing to beta-test anything you do as I have the CTK6200, PX575, PX560, PX350 and XW-P1 although the XW also has a very developed software front end for it, so if these have that level of midi implementation/accessibility, looking at the other implementation charts, I'm sure other Casios can be accessed as well at least to some degree. Thanks for reading, and sorry if I'm diverting from your original project, will drop it if it is an imposing question! If I had the Python chops (I'm working on it) I'd help develop this myself. 

[shiihs]

16 minutes ago, Jokeyman123 said:

Do you think it is at all possible with your programming skills to develop some type of software editor for any of the Casios that do not have this? This would include specifically the CTK6000, 6200, 6250-7000, 7200 and the WK6500, 6600, 7500 and 7600 which probably could do with one software program for all of these with some slight variations-the drawbar functions that are not a part of the 6200, 6500 etc. 

 

Well... there are multiple aspects to your question.

 

If you are talking about making a software front-end that is communicating directly with the Casio keyboard, that would potentially be a little simpler to make (but a ton more work!) because this can be done by sending sysex midi messages to the Casio keyboards, and then saving the configured tone on the keyboard itself. The key point here is that these sysex midi messages are documented, so there's a solid foundation to start from. I think this is exactly what @markonis is working on at the moment, and I don't want to duplicate his efforts (I've had a quick look at the sysex documentation, and it's a LOT of work to support everything that is documented).
 

If you are talking about an editor that can change the contents of the binary file formats that Casio saves without having access to a keyboard (like .tn7 or .ac7 or whatever else they produce), then it is more difficult because trying to figure out the meaning of the information in the binary file formats is a pretty painful process (it's completely undocumented). The programming language in this case really is the least of the obstacles to overcome.

 

My own motivation for working on the binary .ac7 files stems from a larger goal of mine, which (as a sub-task) involves being able to generate style files from computer code (as opposed to recording them by live playing on the keyboard). I'm not there yet, so for now I prefer to keep concentrating on proceeding with what I started. 

 

[Jokeyman123]

I was not aware of Markonis work on this, great.  As far as deciphering Casio's binary codes, understood. Thanks for the response. Got it.

[markonis]

@Jokeyman123: As you've correctly pointed out the deep built-in tone/dsp editing functions available on WK / CTK models are not easily editable through the keyboard. There are multiple pages per tone and only few parameters on a page can be edited at one time. That was the main reason for me to start creating the software that would make this process easier.
When I had the basic GUI for tone editing feature ready it kind of reminded me a standard VST plug-ins which are freely available on the web. The difference is that VST plug-ins try to be universal and thus they offer "only / mostly" standard MIDI functionality ( i.e. standard sysex messages ).
What I'm trying to build is a custom made tool specifically tailored for CASIO WK/CTK models ( i.e. using also provider-specific sysex messages which would enable us to use maximum of features / functionality of a keyboard ).
As @shiihs correctly points out the MIDI features implemented in CASIO keyboard are documented quite well in the specification but it's a lot of work to extract and convert inner data structures used by the keyboard to GUI controls.

There are few obstacles in this process that I'm currently dealing with:

1) the CASIO allows to send only user tones - not built-in tones ( note: each user tone is based on one built-in tone ).
2) the CASIO sends parameters for a tone as a block of bytes. There are few parameters that are common for all tones, but many tones ( mostly dependent on DSP settings ) have variable structures ( for ex. different number of parameters per tone, different number of bytes per parameter, ... ). 

3) as a result of an obstacle 2) the number of bytes received per tone ( = number and values of tone parameters ) differ for each tone

4) CASIO doesn't send a specific sysex just for a movement of a slider / knob, it sends it only as a reaction to change of some tone's parameter 

      => I need to ensure that the keyboard is set in a special way - as if a user changes some parameter ( let's say tone velocity ) with the slider.

      => but, at the same time I need to ensure that a correct tone parameter is changed in the tool as a response to a slider / knob move and that GUI reflects the change properly.

So, to overcome the obstacle 1) and get the proper list of parameters for a built-in tone I need to manually change some parameter of a built-in tone on the keyboard and save this changed tone as a user tone - this way I can export all the parameters for any tone  ( built-in or user tone ).
 

The more difficult issue are the obstacles 2) and 3) - in order to find out a specific parameter of a tone ( + where it's located in the array of exported bytes ) I need to change the relevant parameter on the keyboard, save the tone, export it and compare all the changed bytes with the original tone parameters list ( bytes ).

In the attached screenshot you can see that number of Tone Basic parameters 1 and 2  ( + Tone DSP parameters 1 ) is fixed but the number of Tone DSP parameters 2 is different for each tone and at this time I don't have the map available for all the tones and it's ( variable ) parameters. Of coarse, these variable tone parameters and its values need to be generated dynamically in the tool ( when the tone is selected ).

If we take into account that there are 700 built-in tones it's obvious that the number of combinations of tones and its parameters is quite huge 🙂 
( as a side task, I'm working on a way how to automate an extraction and a proper mapping of these parameters ).

 

The good think is that I have ( roughly ) finished the framework so that I can now concentrate on the first module's  ( = Tone Editor ) main functionality.
Your comments are welcome.

[Jokeyman123]

Wow, what an elegant front end-what amazing work. I had long been puzzled by how certain Casio tones embed DSP settings as part of the tone, which is quite a bit different than any other synth structure I've worked with, so I clearly understand this is a rather difficult design to deal with, even when trying to edit a tone from the keyboard-creates confusion (IMO) at the user level and so I recall on reading many posts here. This seems to be the design used by the PX350 piano, the PX560 and my little CTK6200, I'm not sure about the older PX575-which has almost identical architecture as many of the earlier WKs-the 3000, 3300, 3600 and even the WK8000 88-key, a rather more obscure Casio. 

 

One other odd design I've puzzled over-regarding specifically the PX350, which has no editing facilities at all-and yet boasts a 1 17-track sequencer and a very decent array of tones/rhythms/registrations etc.-studying the midi implementation chart for this one reveals many functions one would think could be user accessed with the right software-even though changes probably cannot be saved, and I suspect many of the other Privias are somewhat similar-many available sysex messages that could make this so much more than it is-as if Casio intended a software front-end for this, and decided to go with the large screen rather than developing the software-very strange since again, the IDES 4.0 is a full software suite for editing tones, samples, DSP settings, organ drawbars and a few other features that I don't have in front of me. I will be glad to "beta test" your software with the CTK6200 and the PX350-jus to see if that monster will recognize anything at all! Not being a programmer and software designer, I have been very limited in what I could do. Tnicoson si another poster here who is very knowledgeable regrading the WK/CTK functions (sorry T if you wanted to remain anonymous!) 

 

One final thought-your efforts should not go unrewarded (IMO). At some point if you will accept small donations-not here, commercial enterprise is not allowed here, understandably-but as many open-source developers ask for contributions-if you post the software elsewhere-I will be happy to donate something. This is looking like a really nice Casio-friendly software program. 

 

PS-curious, what are you using to develop this-shiihs is using Python which I am now studying based on shiihs' development.

[markonis]

I like to develop SW in Visual Studio - my favorite languages for Windows / web applications are C# / ASPX.NET.
Within a data warehouse projects ( which is my core business ) I work with MS SQL Server and Oracle databases ( i.e., SQL and T-SQL languages  ).

[markonis]

32 minutes ago, Jokeyman123 said:

Wow, what an elegant front end-what amazing work. I had long been puzzled by how certain Casio tones embed DSP settings as part of the tone, which is quite a bit different than any other synth structure I've worked with, so I clearly understand this is a rather difficult design to deal with, even when trying to edit a tone from the keyboard-creates confusion (IMO) at the user level and so I recall on reading many posts here. This seems to be the design used by the PX350 piano, the PX560 and my little CTK6200, I'm not sure about the older PX575-which has almost identical architecture as many of the earlier WKs-the 3000, 3300, 3600 and even the WK8000 88-key, a rather more obscure Casio. 

 

One other odd design I've puzzled over-regarding specifically the PX350, which has no editing facilities at all-and yet boasts a 1 17-track sequencer and a very decent array of tones/rhythms/registrations etc.-studying the midi implementation chart for this one reveals many functions one would think could be user accessed with the right software-even though changes probably cannot be saved, and I suspect many of the other Privias are somewhat similar-many available sysex messages that could make this so much more than it is-as if Casio intended a software front-end for this, and decided to go with the large screen rather than developing the software-very strange since again, the IDES 4.0 is a full software suite for editing tones, samples, DSP settings, organ drawbars and a few other features that I don't have in front of me. I will be glad to "beta test" your software with the CTK6200 and the PX350-jus to see if that monster will recognize anything at all! Not being a programmer and software designer, I have been very limited in what I could do. Tnicoson si another poster here who is very knowledgeable regrading the WK/CTK functions (sorry T if you wanted to remain anonymous!) 

 

One final thought-your efforts should not go unrewarded (IMO). At some point if you will accept small donations-not here, commercial enterprise is not allowed here, understandably-but as many open-source developers ask for contributions-if you post the software elsewhere-I will be happy to donate something. This is looking like a really nice Casio-friendly software program. 

 

PS-curious, what are you using to develop this-shiihs is using Python which I am now studying based on shiihs' development.

I'm afraid that at the start of the beta-testing we will need to concentrate on WK 7500/7600 and CTK 7000/7200 models since MIDI implementation of sysex messages seems to be the same. The main differences between the above models ( from the tool's perspective ) are "only" the number of keys and a user storage available per data structure ( user tones, user DSP effects, reg. bank slots, ... ).

As I already quickly studied MIDI implementation of some other CASIO models I can see that the structure of sysex messages differs per model. I'm building the framework so that future expansion / incorporation of other models is possible but first I need to test the functionality on a keyboard that I have at home ( WK 7600 ).
That doesn't mean that we can't test the tool on other models but I'm 90% sure that it'll not work.
The worst case will be if internal structures of CASIO objects will differ - in that case I will need to adjust most of the framework, i.e. make it generic. Which is doable, but definitely another project.

 

[markonis]

10 minutes ago, markonis said:

I'm afraid that at the start of the beta-testing we will need to concentrate on WK 7500/7600 and CTK 7000/7200 models since MIDI implementation of sysex messages seems to be the same. The main differences between the above models ( from the tool's perspective ) are "only" the number of keys and a user storage available per data structure ( user tones, user DSP effects, reg. bank slots, ... ).

As I already quickly studied MIDI implementation of some other CASIO models I can see that the structure of sysex messages differs per model. I'm building the framework so that future expansion / incorporation of other models is possible but first I need to test the functionality on a keyboard that I have at home ( WK 7600 ).
That doesn't mean that we can't test the tool on other models but I'm 90% sure that it'll not work.
The worst case will be if internal structures of CASIO objects will differ - in that case I will need to adjust most of the framework, i.e. make it generic. Which is doable, but definitely another project.

 

Correction: it seems to me now that a list of models supported by the current version of a tool is broader - CTK6200 / CTK 7300 / CTK 7200 / CTK 7300 / WK 6600 / WK 7600.

[shiihs]

10 hours ago, markonis said:

Your comments are welcome.

 

 

 

It looks quite fancy already. From what I've read, different IDES versions are really tailored to specific models of CASIO keyboards, so that may be an indication that the internal structures between these models are quite different. Your tool being written in C#, I'm not sure how easy it will be to test on Linux. I know there's mono (plagued with bugs from what i read, and .NET core (but most likely you will rely on extensions that are not supported on Linux).

 

Just one more remark: please start a new topic for further discussion about your tool. Let's keep this topic on-topic.

 

[Jokeyman123]

Right-these 2 projects are each important enough to keep in their own unique posts-I'm sure many here will be very happy to see both these projects being developed, and feedback and questions could become quite involved for both!

[markonis]

@shiihs I hesitated if I should reply and post a comment on your topic but then the discussion really went off this topic. Sorry for that, I didn't mean to advertise my tool here. Few month ago I've started a topic for my tool (  WK 7600 - Structure of file formats ( i.e. AC7, TN7, RM7, AL7 ) in the form of a question / request for a help. Is it possible to move the above  conversation to my topic?
Regarding the portability to Linux: I'm a whole-life Windows developer so Linux is unknown to me. From what I've read about Mono and  NET Core I would probably vote for Mono with a thorough testing. However, since I'm dealing with a low-level functionality strongly dependent on a HW device plus specific user GUI biased to this HW functionality I see it very difficult for me at this time. I have some ideas on how to do it but I agree that we should discuss it within my topic.

[Chandler Holloway]

I’m sure @Brad Saucier could move your conversation to the appropriate topic. He’s very good about keeping things organized around here.

[Brad Saucier]

I think it's best to leave everything where it's at for now.  If I move some of these posts to the other forum, the conversation may appear incomplete to anyone not familiar with what is happening here.  Venture over the other thread to continue discussion about @markonis work.  

 

https://www.casiomusicforums.com/index.php?/topic/16912-wk-7600-structure-of-file-formats-ie-ac7-tn7-rm7-al7/