BAAuto & Ideas

[RedDeadDepresso]

Hi, I tried AAS, and I'm truly impressed by how you managed to adapt ALAS for Blue Archive.

I've developed a script for Blue Archive myself, called BAAuto. However, it's not very optimised and tends to run slowly.

I believe there are some promising ideas in my script, and I'm more than happy to offer the source code or any other elements for your use. However, please keep in mind that the source code might be messy at times, as I haven't had the chance to refactor it.

You can check out the detailed information I've provided about BAAuto in this GitHub issue, and if you have any questions, I'm more than willing to answer them.

Also, if you can implement my Mission/Commissions/Event idea, I'm open to adapting AAS for EN when I have the time.

[TheFunny]

Hi, I tried AAS, and I'm truly impressed by how you managed to adapt ALAS for Blue Archive.

I've developed a script for Blue Archive myself, called BAAuto. However, it's not very optimised and tends to run slowly.

I believe there are some promising ideas in my script, and I'm more than happy to offer the source code or any other elements for your use. However, please keep in mind that the source code might be messy at times, as I haven't had the chance to refactor it.

You can check out the detailed information I've provided about BAAuto in this GitHub issue, and if you have any questions, I'm more than willing to answer them.

Also, if you can implement my Mission/Commissions/Event idea, I'm open to adapting AAS for EN when I have the time.

First of all, thank you for trying AAS. I'm actually surprised of how fast that anyone would try this out and write a reply to me because I've just uploaded the code last week and updated English GUI support yesterday. I would appreciate it if you can report any problem of using the script, including bugs,

As for BAAuto, I will check that out whether I can implement your ideas in AAS in my spare time. I'm currently struggling on how to combine multiple server support with OCR on specific keyword (e.g. School names in Schedule, student names, shopping, etc.).

Please feel free to make any contribution to this project.

[RedDeadDepresso]

I think the best solution would be directly contacting LmeSzinc, ALAS developer, as they have experience in this matter.

Otherwise, my solution for your issue would be like this. Let's consider shop. My approach would be creating a directory named 'shop,' which will contain multiple JSON files. Each JSON file should be named after the respective server, such as 'EN.json,' 'JP.json,' 'CN.json,' etc. These JSON files will contain dictionaries that map item names in English to their corresponding translations in the server's language. For instance:

{
  "Novice Activity Report": "Translation in Japanese",
  "Normal Activity Report": "Translation in Japanese",
  ...
}

You can then easily perform OCR like this:

with open(f"{server}.json", "r") as f:
    items = json.load(f)

find(items["Normal Activity Report"])

To obtain a JSON file containing the names of all items, you could either perform web scraping using the python libraries BeautifulSoup and requests or search for repositories on GitHub that contain them. For web scraping and if the data is stored as a table, the easiest way would be to use Excel 'get Data from web', export it as csv or .txt file and then convert it to json using python.

I have some json files for student names if you need them, however they are stored as a list.
https://github.com/RedDeadDepresso/BAAuto/tree/main/gui/student_list

Here is a website that displays the shop items in English:
https://bluearchive.wiki/wiki/Shop

[RedDeadDepresso]

For student names, If you only want to invite students in the cafe, my approach in BAAuto was as follows: I created a combo box that dynamically displayed student names based on the server language. For instance, if the server was set to CN, the combo box would show Chinese names, and if it was set to EN, it would display English names. Users still had the option to manually enter a student's name if needed.

The algorithm I used for this task is quite straightforward:

1. I use template matching to see if the student list appeared and retrieve the student name from a configuration file.
2. Subsequently, I crop the 'momotalk' region, specifically the area defined by coordinates (410, 187, 700, 600).
3. Next, I utilise OCR to read the names. If the desired name is found in the list, I resize the 'invite' button's height if needed, locate the closest one and click on it. Otherwise, I store the last name on the list and swipe from (600, 500) to (600, 200).
4. If the last name reappears, it means the end of the list, so I notify the user that no student was found and to check the spelling.

Here is the source code if you need it.

[RedDeadDepresso]

Also, for Bounty and Scrimmage, I would recommend using numerical stage designations instead of alphabetical letters, such as '01' instead of 'A.' This is because the positions of the number is always consistent between different languages.

To read numbers like AP, stage, area, or sweep counter, use the pponnxcr en OCR model as it is more accurate.

[TheFunny]

1. As for multiple server / language support, the idea I've come up with is quite similar with yours. My idea is to maintain two sets of files. The first set of files is to store a general list of keywords. These keywords will be used to generate the second list of files, where the files will be separated into different languages. Each language file contains keywords with their translation (to be added by developer later), which can be used in OCR. The reason why I would like to choose this structure is that developers can quickly add new keywords in the first set of files and then generate the second list of files. The missing keywords will be added while those keywords that already have translation will be kept. Finally, the second list of files will be used to generate a list python files, including keywords variables (most of the codes of modules in this project are from SRC, and it has a keyword class to be used in OCR). Each keyword variable is a keyword class that contains all server translations of this keyword. Thus, we can directly use the keywords when writing new codes without worrying about multi-server problems.
2. The strategy you use to choose student names is quite interesting, but it can be difficult for AAS because it’s ALAS based framework have restricted GUI options. However, ALAS has a filter system that allows user to decide priority of student invitations. The solution I would suggest is to manually input student names combined with the filter system. For the algorithm you use for invitation, it’s effective and I will consider implementing it. Thank you for providing it.
3. I will definitely use numerical recognition whenever possible due to its simplicity and stability. Thanks for your advice.

(My English is not so good but I hope I've made my explanation clear)

[RedDeadDepresso]

I clearly understand what you mean and you are also more familiar with the ALAS framework than me so I think your solution would be the best.

Also to be more specific for finding the button:

1. Identify the text region of interest e.g the stage number, resize the button template's height with the height of the text region.
2. Take a screenshot and crop it with the top-left corner coordinates same as with the text region's top-left coordinates. The width of the crop the same as the search region width e.g the stage list.
3. Do template matching on the crop. If the button is found then click on it, otherwise the stage was not unlocked.

This would be the simplest approach, however, my implementation in BAAuto is more reliable but slower.
https://github.com/RedDeadDepresso/BAAuto/wiki/Finding-Button

[RedDeadDepresso]

Hi, I thought it would be difficult and time consuming for you to implement Mission/Commissions/Event, so I isolated it from BAAuto and now you can use it for AAS. You just need to read MCE\config.json and implement the algorithm.

https://github.com/RedDeadDepresso/MCE-Manager

This is just a temporary solution, and it would be better if you implement it in AAS GUI later yourself.

[TheFunny]

I've implemented functions of interacting with stage list in the latest commits of dev branch, which supports finding specific index number and clicking corresponding enter button. The approach of finding the desired button is originated from the solution you provided. It seems working fine in early test, and I will try implementing Mission in a few days.

[RedDeadDepresso]

It's nice to hear it. I was looking at the assets and found that there are about 100 of them. It would take too long and I'm busy at the moment. I plan to begin adapting AAS to English in mid-December when I'll have more free time. Also, I only realised now what you mean by circle and schedule. In the English version they were translated as club and lessons respectively. I attached the games homepage and campaign page to help you familiarise with the names.

[RedDeadDepresso]

Also, I didn't plan to implement lessons as I believed it would be more effective for users to manage it themselves. However, since you plan to implement it, I'll share how I would approach it. Consider Mission/Commissions/Event but with an interface comprising:

• Up button
• Down button
• School name
• A checkbox indicating whether it's enabled or not
• The bottom buttons being 'Select All' and 'Clear All' which select and deselect all the schools respectively

The representation would be a list of lists in the configuration file, for example:

[[school1, False],
[school2, True],
...]

Additionally, there's a dropdown menu named 'if remaining tickets' with school options.

Here's the outlined process:

1. Iterate through the list of lists. If a school is enabled, go to that school and click on the bright red hearts. If not enabled, skip it. Break from the loop if the tickets are 0.

2. If there are remaining tickets (which is unlikely), go to the school selected from the 'if remaining' dropdown. Utilise all tickets until they reach 0 or the entire area is shaded black.

The checkbox serves a purpose. It allows users to specify if they've unlocked the school or if they prefer not to spend any tickets on it. Users can also concentrate all their tickets on one school by disabling others and selecting the desired school in the 'if remaining tickets' dropdown.

I have attached images of the school names in English. The locked ones are Red Winter Federal Academy and Hyakkiyako Central Area.

[RedDeadDepresso]

If you intend to implement Shop, Momotalk or Crafting, I have no idea. You can get inspiration from pur1fying's script or https://github.com/baas-pro/baas, which I found recently. I tried to contact its developer to help with AAS but they didn't give any response.

[TheFunny]

1. For the assets, the steps for creating assets is provided here. The guide is written in Chinese, and here is an English version if you need:
   a. Take a screenshot with the internal capture tool provided by emulator. Ensure its resolution to be 1280x720
   b. Copy the screenshot to the corresponding directory in assets and rename it
   c. Use Photoshop to extract the desired button area from the screenshot. The rest part of the file should be in black. Here it utilize the shortcut in PS, which supports recording certain actions so that we can reuse them
   d. Those files ending with .BUTTON are used to overwrite the button property. For example, A.BUTTON.png will overwrite button property of A.png. This is for clicking a different area other than the recognition area
   e. Run python -m dev_tools.button_extract
   You may just copy the jp folder and rename it to en, then substitute those files with EN server ones. For some files that can be shared between different servers, you can keep them the same, and I will deal with that later.
2. Thanks for providing translations in English. I will adjust text in GUI for better accuracy and understanding.
3. That is quite helpful. I will try implementing this. Thanks for the idea.
4. The implementation of Shop, Momotalk and Crafting is planned but with a low priority. Thanks for sharing that and I will check that out after completing the work I'm now busy with.

[RedDeadDepresso]

Thank you for your help; you've clarified some of my doubts. Please don't feel obligated to rush and take your time, I'm just offering ideas and guiding you towards resources to assist you.
Additionally, I need to correct a point from the lessons; it wasn't shaded area, but rather, all the student icons were shaded black.
Regarding movement, I've considered a square or pentagon motion as the most effective. The approach involves creating a list of swipe sequences defined by the polygon's shape.

So for step 2,

1. iterate through the list of swipe sequences, and for each iteration, perform a swipe using the coordinates specified in the list.
2. Implement an object detection algorithm to identify all students and click on those that are not shaded black.
3. The stopping conditions should be either when the available tickets reach 0 or when you've completed iterating through the list of swipe sequences.

I think you can reuse this function to check if the student icon is shaded black:

  def is_obscured(cls, location, template_shape):
      """Check if the template match is obscured by surrounding pixels.

      Args:
          location (tuple): Location of the template match.
          template_shape (tuple): Shape (height, width) of the template.

      Returns:
          bool: True if obscured, False otherwise.
      """

          # Extract the region around the template match
          x, y = location
          region = cls.color_screen[y:y+template_shape[0], x:x+template_shape[1]]

          # Calculate the average darkness level of the region
          avg_darkness = numpy.mean(region)

          # You can adjust the darkness threshold as needed
          darkness_threshold = 100  

          return avg_darkness < darkness_threshold

[RedDeadDepresso]

I followed the instructions and got these results for club. Are they fine? I used the quick selection tool for the confirm button and I'm not sure if the "Club Attendance Rewards" text is allowed to be long.

[TheFunny]

1. For lessons, do you mean directly clicking on the map of school for selection? I think a simpler way to deal with lessons is using this page:
   
   I can give user choice to either select area with heart or select specific area. Thanks for providing the function and I will see if that can be applied.
2. For the first button, you can just select the inner area in a rectangular shape, and it will work fine. I'm not sure if a shape of parallelogram will cause any problem, but the rectangular ones in jp assets work. So I think it's better to just use rectangle button. For the second one, it should be fine since the script directly compares the button with the corresponding region in screenshot using template match.

[RedDeadDepresso]

1. My mistake, I hadn't noticed that page. Yes, you should use it because it's easier.
2. Thanks for the clarification. In the end, I used a rectangle shape for the buttons to avoid any problems.

[RedDeadDepresso]

I only need to make popup assets but I need some clarification. Can you explain where do you get:
GET_NEW_STUDENT,
NETWORK_RECONNECT_OK ,
AP_EXCEED.
Also, can you explain how do you run AAS with GUI from the source code? I'm not finding any information in the Alas documentation.

[TheFunny]

1. GET_NEW_STUDENT is displayed when getting student, for example, when you complete the Story of Event. This is designed to automatically complete Event. However, it seems that the region I’ve extracted won’t display when getting a new student, and I consider finding another solution so this button will be removed.
2. NETWORK_RECONNECT_OK is displayed when the game is attempting to connect to server but fails. I think the possibility that this button is shown can be quite low because I’ve also never seen that, and I got this screenshot from my friend, who has a very poor network connection. I’ve only seen NETWORK_RECONNECT, which is shown much more frequently. Btw, since none of JP nor any Global server accepts connection of IP from China, so we need some techniques to play the game, and that also introduces possibilities of network problems. Therefore, this can be temporary ignored if you never encounter that.
3. AP_EXCEED is displayed when user attempt to claim AP after it has reached 999. This popup tells user that the exceeded part of AP is send to mailbox. I think this popup will also be shown when any other item reaches its limit, but that happens extremely rarely.

[TheFunny]

For running AAS GUI from source code, firstly you need to install the requirement. However, some of the auto generated requirements is in incorrect version so the script doesn’t work if directly using that. It’s a pity that I haven’t got time to write a detailed document for running from source code. I guess the script could run under environment that provided by the toolkit (within the installation of Released version), so you may try that out. If that doesn’t work, I will check the correct version of the modules that cause this problem.
After that, you can run python gui.py in root directory. The script will open a local http server for webgui. You can use this url http://127.0.0.1:22367

[RedDeadDepresso]

Thanks, I was able to test it, and everything works great. The only problem is that it's stuck on this page.

Also, let the user know that Second Cafe only works for JP.

[RedDeadDepresso]

I finished most of the popup assets; the only thing left is the 'ITEM_EXPIRED' popup, and I have to wait six days since I always claim rewards from the mailbox. If you want, I can make a pull request since it's pretty rare to encounter, and AAS can work without it.

For NETWORK_RECONNECT_OK, I found this to be the closest. Feel free to correct me.

For the UPDATE in login, I had to use "Ready to download" as text because I had two types of popups, one for essential files and the other for additional files.

[RedDeadDepresso]

For the lesson, to navigate, I would go to the lesson page and then scroll up and click on the first school. I would then create a list of school names images, iterate through it, and find which one matches the current school name in the top-right corner. After that, I would determine which direction to click (left or right), by checking if the current school index is greater or less than the target school index. I would click as many times as the difference in index between the two.

I believe this is the implementation of pur1fying based on my reading of the source code, but I'm not entirely sure. This approach is better because template matching is faster and more reliable than OCR. For the rest, you can implement whatever you prefer.

[RedDeadDepresso]

I want to suggest for the GUI to create a tree view Farming and Claim Rewards with the same options as BAAuto. Also, would it possible to hide Dashboard Upd since the user cannot modify it?

[RedDeadDepresso]

Aditionally, I was thinking of a feature called Auto-Event. While playing through the new event story and quest, I noticed that it was mainly about waiting and clicking buttons. Since you have already implemented searching stages, it should be straightforward. The events don't have tile gameplay. I used BAAuto to sweep through many events, and the only thing I had to change was the event banner. This feature is pretty much similar to Continuous Clear in Alas.

Now, let's delve into the GUI options:

• Explosive Unit, Piercing Unit, Mystic Unit, each with a corresponding dropdown featuring choices from Unit1 to Unit6.
• RechargeAP, the same as in Mission/Commissions/Event.
• Level achievement with a dropdown offering options: clear or 3 stars.
• A checkbox for event story, indicating whether it's enabled, accompanied by a dropdown menu ranging from 01 to 08.
• A checkbox for event quest, with a dropdown menu presenting options from 01 to 12.

Here's how it functions:

1. Navigate to the designated level from the dropdown.
2. Check if it meets the level achievement. If not, click the enter button.
3. Click on the Enemies button and count the number of enemies weak to each type using template matching.
4. Identify the type to which most enemies are weak and select the appropriate unit from formation.
5. Click the auto button and wait for completion.
6. Repeat this cycle until reaching the conclusion of possible stages, a defeat or not meeting the level achievement after running the stage.
7. Optionally, recharge AP if it gets depleted.

[TheFunny]

1. You can add an asset to handle this survey page. I suggest using the cross in the top-right corner. Since I've never seen this page on JP server, I guess it's Global server only. I'll add a function to handle this. For the second floor, I've added the function to detect whether it's JP or not and also the description to notify user, but I haven't push the commits.
2. Yes. For most of the time, AAS can handle this. For other buttons, you are free to adjust region to be recognized as long as the logic for handling them are the same and the buttons can work without any problem.
3. Thanks for the advice. I agree with that since it would be easier to be implemented and maintained with more reliability.
4. I do want to use the interface like BAAuto because it's more friendly to users while providing with flexibility. It's a pity that the GUI of ALAS may not be able to complete that due to its restrictions. I'm not familiar with coding of GUI, so the solution for now is to use the idea like filter in ALAS. For hiding the Dashboard Upd, it's possible and I'll finish that later.
5. The information is really helpful for implementing Auto-Event. I will refer to this when working on that.

If every thing is working fine with EN, you can open a PR after sufficient tests.

[RedDeadDepresso]

1 - I've already included the survey popup in the assets but I was thinking that would it not be possible to simply click on the top right corner if an unknown page is found when running login? All login popups have a cross there.

5 - I noticed that aas.py is the script and the GUI is modifying config/config.json. Since this approach is very similar to BAAuto, I could adapt it and provide it as a second GUI, but some options would be hidden from the user and the language would only be English. Also, I'm currently busy right now so it's going to take some time.
The only thing left to know is whether AAS always has config.json open and if so what would happen if BAAuto tries to modify it.

[TheFunny]

1. It's possible but I'm afraid that it may cause unexpected problems. Because of the movie in the background of login page, it would be difficult to identify a certain page. It may result in a continuous click on the same button that the script will throw an exception on this. For now, I'd choose the original method.
2. It's ok if that is possible. You can take your time as it's not urgent. I will see what I can do after implementing Bounty and Scrimmage.

[RedDeadDepresso]

Hi, I've been very busy this week, so sorry I couldn't help you with the bounty and scrimmage assets. I'll be busy in the next few weeks too, so you should ask someone else for EN assets.

I tried Bounty and Scrimmage and was impressed, it was twice as fast as BAAuto. However, there is a bug in Scrimmage. If you have 21 tickets it tells you invalid tickets.

2023-11-23_aas.txt

[TheFunny]

Thanks for reporting bug. The latest commit should have solved this issue.