Implement a new system (still scraping) #4

Open
opened 2025-05-17 23:36:48 +02:00 by val · 1 comment
Owner

"Let's add HEBE routes" - still not done.

In all seriousness though, there is a new way of getting data from VULCAN, and it's only for schools/accounts that migrated to eduVULCAN, but allow for use of the previous way of logging in, ie. through the school link, login and password.

So how does it work?

Logging in (scraping part)

Heading to the usual login page we are met with this:
{7AF113AD-248F-44AC-9AC4-83CB9B469674}
which replaces it.
The URL is: https://uonetplus.vulcan.net.pl/{symbol}/

We have these options:

  • Use the old login system (Logowanie przed zmianą)
    This uses CUFS, which is one of the login types specified in the SDK (sdk-scrapper).
    The URL is: https://cufs.vulcan.net.pl/{symbol}/Account/LogOn?ReturnUrl={usual wsignin login}
    If the student hasn't updated his account to the new system yet, but the school uses the new system, they can still sign in just fine using the SDK.
  • Log in with eduVULCAN (lol no)
  • Log in with the new login system (Logowanie zwykłe konto szkolne)
    This uses a new page, with one input per page (can be skipped)
    The URL is: https://dziennik-logowanie.vulcan.net.pl/{symbol}/Account/Logon?returnUrl=
    Note the new URL, this MIGHT vary per school (not sure)
    This is actually a redirect, just like with the old system.
    I will explain this approach further below.

What are the redirects?

First redirect (button press):
https://dziennik-uczen.vulcan.net.pl/{symbol}/LoginEndpoint.aspx

Second redirect (from the URL):
https://dziennik-logowanie.vulcan.net.pl/{symbol}/Fs/Ls?wa=wsignin1.0&wtrealm=https://dziennik-uczen.vulcan.net.pl/{symbol}/LoginEndpoint.aspx&wctx=auth=student&nslo=1
This redirect sets these cookies:

  • UonetPlus_ASP.NET_SessionId=; domain=vulcan.net.pl; expires={date}; path=/
  • EfebSsoCookie=null; domain=.vulcan.net.pl; expires=Fri, 31-Dec-1999 23:00:00 GMT; path=/; secure; HttpOnly; SameSite=None
    At this request, we get a later needed token called __RequestVerificationToken={value} in the cookies.

Third redirect (from the URL):
/{symbol}/Account/Logon?returnUrl=http://dziennik-logowanie.vulcan.net.pl/{symbol}/Fs/Ls?wa=wsignin1.0&wtrealm=https://dziennik-uczen.vulcan.net.pl/{symbol}/LoginEndpoint.aspx&wctx=auth=student&nslo=1
This has the root URL of https://dziennik-logowanie.vulcan.net.pl/

This is where we log in.

You can visit the login page by visiting any school that uses the new system, so a URL like https://dziennik-logowanie.vulcan.net.pl/warszawa/Account/Logon works just fine.

The script responsible for getting the verification token, along with some parsing info is located at
https://dziennik-logowanie.vulcan.net.pl/Scripts/logon.js?_=25.2.5.61914

I will leave information about how to get said stuff later in the thread.

Logging in (the logic)

We first have to query the user (login) with a POST request to https://dziennik-logowanie.vulcan.net.pl/{symbol}/Account/QueryUserInfo, that contains all our cookies, as well as this body:
login={username}&__RequestVerificationToken={token}&nonce={generated nonce&nonceSignature={nonce signature}
which responds with
{ "success": true, "data": { "ShowCaptcha": false } }.
I suppose we should be ready for handling a captcha on login.

Once the query returns a success, we log in with our login, password, "captcha user" and the request verification token. To do that, we send a POST request to https://dziennik-logowanie.vulcan.net.pl/{symbol}/Account/Logon?returnUrl=http://dziennik-logowanie.vulcan.net.pl/{symbol}/Fs/Ls?wa=wsignin1.0&wtrealm=https://dziennik-uczen.vulcan.net.pl/{symbol}/LoginEndpoint.aspx&wctx=auth=student&nslo=1 with the body: Haslo={password}&captchaUser={empty}&Login={username}&__RequestVerificationToken={token}. This leads us to our fourth redirect.

Fourth redirect (from the login request):
http://dziennik-logowanie.vulcan.net.pl/{symbol}/Fs/Ls?wa=wsignin1.0&wtrealm=https://dziennik-uczen.vulcan.net.pl/{symbol}/LoginEndpoint.aspx&wctx=auth=student&nslo=1
This redirect also sets our cookies:

  • Vulcan.Efeb.Logowanie.Web={long token}; path=/; secure; HttpOnly; SameSite=None
  • Vulcan.Efeb.Logowanie.Web1={another token}; path=/; secure; HttpOnly; SameSite=None

We then do a GET of the redirect, which has a hidden form (wsignin, like in other ways of signing in).
This gives us two new cookies:

The form looks like this:

  <form method="POST" name="hiddenform" action="https://dziennik-uczen.vulcan.net.pl/{symbol}/LoginEndpoint.aspx"><input type="hidden" name="wa" value="wsignin1.0" /><input type="hidden" name="wresult" value="{large encoded XML stuff, the usual wsignin}" /><input type="hidden" name="wctx" value="auth=student&amp;nslo=1" /><noscript>
      <p>Script is disabled. Click Submit to continue.</p><input type="submit" value="Submit" />
    </noscript></form>
  <script language="javascript">
    window.setTimeout('document.forms[0].submit()', 0);
  </script>

Inside the wsignin, there's:

  • an issued token
  • creation date and expiration
  • system: local_sts
  • name: {login}
  • ProfilePrincipal: Unique user ID
  • ClientInstance: {symbol}
  • Id: some long id
  • EfebStartUrl: URL that returns user to dashboard

Fifth redirect: Since the form does a POST request, and it's wsignin, I won't really go in-depth about how to do it, it's automatic I think. It does redirect us to /{symbol} though.
It also sets these cookies:

  • EfebSsoCookie={some long id}; domain=dziennik-uczen.vulcan.net.pl; path=/; secure; HttpOnly; SameSite=None
  • UonetPlus_SLACookie={short id}; domain=vulcan.net.pl; path=/; HttpOnly
  • UonetPlus_ASP.NET_SessionId={session id}; domain=vulcan.net.pl; path=/; HttpOnly; SameSite=Lax

Sixth redirect: Just /{symbol}/App. No cookies.

This is the last redirect that we get, after that we're in the user panel.

So what is actually going on in the panel?

X-V-RequestVerificationToken(hashtag){some id}={long id with a hashtag at the end}; expires={7 days}; path=/; HttpOnly

And the script looks like this:

  <script>
    var VParam = {
      appName: 'Dziennik VULCAN',
      moduleName: 'Panel ucznia',

      antiForgeryToken: '{antiForgeryToken}',
      appGuid: '{appGuid}',

      appUrl: 'https://dziennik-uczen.vulcan.net.pl',
      appCustomerDb: '{symbol}',

      apiUrl: 'https://dziennik-uczen.vulcan.net.pl/{symbol}/api',
      wiadomosciUrl: 'https://dziennik-wiadomosci.vulcan.net.pl/{symbol}/LoginEndpoint.aspx',
      wiadomosciNazwa: 'Wiadomości',
      logoutUrl: 'https://dziennik-uczen.vulcan.net.pl/{symbol}/LoginEndpoint.aspx?logout=true',

      user: {
        name: '{login}'
      },
      serverDate: '{login date, but from the server}',
      appVersion: '{app version, 25.02....}',
      downloaderUrl: 'https://dds.vulcan.net.pl/Download/', // wtf is this
      downloaderBuffer: '500',
      downloaderInterval: '1',

      privacyPolicyUrl: 'https://www.vulcan.edu.pl/strona/polityka-prywatnosci-190',
      respectyPolicyUrl: 'https://dziennik-uczen.vulcan.net.pl/{symbol}/CookiePolicy/SzanujemyPrywatnosc',
      cookieSettingsUrl: 'https://dziennik-uczen.vulcan.net.pl/{symbol}/CookiePolicy/UstawieniaCookie',
      infoEnclosureUrl: 'https://www.vulcan.edu.pl/strona/klauzula-informacyjna-868',
      accessDeclarationUrl: 'https://www.vulcan.edu.pl/strona/deklaracja-dostepnosci-869',
      helpUrl: 'https://www.bazawiedzy.vulcan.edu.pl/bazawiedzy.php/wyszukaj/1/7/8/2/2/',

      manualUrl: 'https://aplikacje.vulcan.pl/uonetplus/DziennikUczen.pdf',
      lastUpdatesUrl: 'https://www.vulcan.edu.pl/strefa-klienta/aktualizacje/E-dziennik-UONET,7',

      footerLogo: '',


      isStudent: true,
      isParent: false,

      isDziennikVulcan: true
    };
  </script>

Well, this is new...

Doing requests

Every API request requires two headers:

  • x-v-appguid: {appGuid in script}
  • x-v-requestverificationtoken: {token from script}

Surprisingly enough, we can get the OLD info fields from another API request called "context", which gives us all the stuff, even the stupid "isArtystyczna13", etc...

The request looks like this:
GET - https://dziennik-uczen.vulcan.net.pl/{symbol}/api/Context
(just our typical cookies from the website, no appguid or requestverificationtoken)

  • EfebSsoCookie
  • UonetPlus_SLACookie
  • UonetPlus_ASP.NET_SessionId
  • Long X-V-RequestVerificationToken from the Set-Cookie header in the panel website
{
  "uczniowie": [
    {
      "idJednostkaSkladowa": {number, low},
      "idDziennik": {diaryId},
      "rodzajDziennika": 1,
      "dziennikDataOd": "2024-09-01T00:00:00+02:00",
      "dziennikDataDo": "2025-08-31T00:00:00+02:00",
      "isUczen": true,
      "isPrzedszkolak": false,
      "isWychowanek": false,
      "key": "{key in string, used in EVERY request as a query parameter}",
      "uczen": "{user's first name}",
      "oddzial": "{class, like 1-5}",
      "jednostka": "{school name}",
      "jednostkaGodzinaOd": null,
      "jednostkaGodzinaDo": null,
      "isDorosli": false,
      "isPolicealna": false,
      "is13": false,
      "isArtystyczna": false,
      "isArtystyczna13": false,
      "isSpecjalna": false,
      "pelnoletniUczen": false,
      "opiekunUcznia": false,
      "wymagaAutoryzacji": false,
      "posiadaPesel": true,
      "aktywny": true,
      "globalKeySkrzynka": "{message globalbox key}",
      "config": {
        "isOplaty": false,
        "isPlatnosci": false,
        "isZaplac": false,
        "isScalanieKont": false,
        "isJadlospis": false,
        "isOffice365": false,
        "isSynchronizacjaEsb": true,
        "isDydaktyka": true,
        "isNadzorPedagogiczny": true,
        "isZmianaZdjecia": true,
        "isZglaszanieNieobecnosci": true,
        "isLekcjeZrealizowane": true,
        "isLekcjeZaplanowane": true,
        "isPodreczniki": true,
        "oneDriveClientId": "{onedrive}",
        "projectClient": null,
        "payByNetUrlForPayment": "{paybynet stuff}"
      }
    }
  ]
}

For things in the API that are fetched at the panel itself, it doesn't use the key, but in fetching grades, etc. it does.

An example is "PlanZajecTablica":

GET - https://dziennik-uczen.vulcan.net.pl/{symbol}/api/PlanZajecTablica?key={key from context}

[
  {
    "data": "9999-99-99T00:00:00+02:00",
    "godzinaOd": "9999-99-99T00:00:00+02:00",
    "godzinaDo": "9999-99-99T00:01:00+02:00",
    "prowadzacy": "{teacher name}",
    "prowadzacyWspomagajacy1": null,
    "prowadzacyWspomagajacy2": null,
    "przedmiot": "{subject name}",
    "podzial": "{group in gr_(number) format}",
    "sala": "{room number}",
    "pseudonim": null,
    "zmiany": [],
    "zmianyUwagi": [],
    "adnotacja": 0,
    "dodatkowe": false,
    "zrealizowane": true,
    "idJednostkaSkladowa": {stuff from context}
  },
  // ...
]

I will continue research / dropping more info in the thread below - along with API requests that I have access to

"Let's add HEBE routes" - still not done. In all seriousness though, there is a new way of getting data from VULCAN, and it's only for schools/accounts that migrated to eduVULCAN, but allow for use of the previous way of logging in, ie. through the school link, login and password. So how does it work? ### Logging in (scraping part) Heading to the usual login page we are met with this: ![{7AF113AD-248F-44AC-9AC4-83CB9B469674}](/attachments/fc9ce5dd-8193-477f-9a6b-88b092d645f1) which replaces it. The URL is: `https://uonetplus.vulcan.net.pl/{symbol}/` We have these options: - Use the old login system (Logowanie przed zmianą) This uses CUFS, which is one of the login types specified in the SDK (sdk-scrapper). The URL is: `https://cufs.vulcan.net.pl/{symbol}/Account/LogOn?ReturnUrl={usual wsignin login}` If the student hasn't updated his account to the new system yet, but the school uses the new system, they can still sign in just fine using the SDK. - Log in with eduVULCAN (lol no) - Log in with the new login system (Logowanie zwykłe konto szkolne) This uses a new page, with one input per page (can be skipped) The URL is: `https://dziennik-logowanie.vulcan.net.pl/{symbol}/Account/Logon?returnUrl=` Note the new URL, this MIGHT vary per school (not sure) This is actually a redirect, just like with the old system. I will explain this approach further below. #### What are the redirects? First redirect (button press): `https://dziennik-uczen.vulcan.net.pl/{symbol}/LoginEndpoint.aspx` Second redirect (from the URL): `https://dziennik-logowanie.vulcan.net.pl/{symbol}/Fs/Ls?wa=wsignin1.0&amp;wtrealm=https://dziennik-uczen.vulcan.net.pl/{symbol}/LoginEndpoint.aspx&amp;wctx=auth=student&nslo=1` This redirect sets these cookies: - UonetPlus_ASP.NET_SessionId=; domain=vulcan.net.pl; expires={date}; path=/ - EfebSsoCookie=null; domain=.vulcan.net.pl; expires=Fri, 31-Dec-1999 23:00:00 GMT; path=/; secure; HttpOnly; SameSite=None At this request, we get a later needed token called `__RequestVerificationToken={value}` in the cookies. Third redirect (from the URL): `/{symbol}/Account/Logon?returnUrl=http://dziennik-logowanie.vulcan.net.pl/{symbol}/Fs/Ls?wa=wsignin1.0&wtrealm=https://dziennik-uczen.vulcan.net.pl/{symbol}/LoginEndpoint.aspx&wctx=auth=student&nslo=1` This has the root URL of `https://dziennik-logowanie.vulcan.net.pl/` This is where we log in. You can visit the login page by visiting any school that uses the new system, so a URL like https://dziennik-logowanie.vulcan.net.pl/warszawa/Account/Logon works just fine. The script responsible for getting the verification token, along with some parsing info is located at `https://dziennik-logowanie.vulcan.net.pl/Scripts/logon.js?_=25.2.5.61914` I will leave information about how to get said stuff later in the thread. #### Logging in (the logic) We first have to query the user (login) with a POST request to `https://dziennik-logowanie.vulcan.net.pl/{symbol}/Account/QueryUserInfo`, that contains all our cookies, as well as this body: `login={username}&__RequestVerificationToken={token}&nonce={generated nonce&nonceSignature={nonce signature}` which responds with `{ "success": true, "data": { "ShowCaptcha": false } }`. I suppose we should be ready for handling a captcha on login. Once the query returns a success, we log in with our login, password, "captcha user" and the request verification token. To do that, we send a POST request to `https://dziennik-logowanie.vulcan.net.pl/{symbol}/Account/Logon?returnUrl=http://dziennik-logowanie.vulcan.net.pl/{symbol}/Fs/Ls?wa=wsignin1.0&wtrealm=https://dziennik-uczen.vulcan.net.pl/{symbol}/LoginEndpoint.aspx&wctx=auth=student&nslo=1` with the body: `Haslo={password}&captchaUser={empty}&Login={username}&__RequestVerificationToken={token}`. This leads us to our fourth redirect. Fourth redirect (from the login request): `http://dziennik-logowanie.vulcan.net.pl/{symbol}/Fs/Ls?wa=wsignin1.0&amp;wtrealm=https://dziennik-uczen.vulcan.net.pl/{symbol}/LoginEndpoint.aspx&amp;wctx=auth=student&nslo=1` This redirect also sets our cookies: - Vulcan.Efeb.Logowanie.Web={long token}; path=/; secure; HttpOnly; SameSite=None - Vulcan.Efeb.Logowanie.Web1={another token}; path=/; secure; HttpOnly; SameSite=None We then do a GET of the redirect, which has a hidden form (wsignin, like in other ways of signing in). This gives us two new cookies: - FederatedLoggedApp_{id}_redirect=https://dziennik-uczen.vulcan.net.pl/{symbol}/loginendpoint.aspx; path=/; secure; HttpOnly; SameSite=None - master_sts=local_sts; path=/; secure; HttpOnly The form looks like this: ``` <form method="POST" name="hiddenform" action="https://dziennik-uczen.vulcan.net.pl/{symbol}/LoginEndpoint.aspx"><input type="hidden" name="wa" value="wsignin1.0" /><input type="hidden" name="wresult" value="{large encoded XML stuff, the usual wsignin}" /><input type="hidden" name="wctx" value="auth=student&amp;nslo=1" /><noscript> <p>Script is disabled. Click Submit to continue.</p><input type="submit" value="Submit" /> </noscript></form> <script language="javascript"> window.setTimeout('document.forms[0].submit()', 0); </script> ``` Inside the wsignin, there's: - an issued token - creation date and expiration - system: local_sts - name: {login} - ProfilePrincipal: Unique user ID - ClientInstance: {symbol} - Id: some long id - EfebStartUrl: URL that returns user to dashboard Fifth redirect: Since the form does a POST request, and it's wsignin, I won't really go in-depth about how to do it, it's automatic I think. It does redirect us to `/{symbol}` though. It also sets these cookies: - EfebSsoCookie={some long id}; domain=dziennik-uczen.vulcan.net.pl; path=/; secure; HttpOnly; SameSite=None - UonetPlus_SLACookie={short id}; domain=vulcan.net.pl; path=/; HttpOnly - UonetPlus_ASP.NET_SessionId={session id}; domain=vulcan.net.pl; path=/; HttpOnly; SameSite=Lax Sixth redirect: Just `/{symbol}/App`. No cookies. This is the last redirect that we get, after that we're in the user panel. So what is actually going on in the panel? Well, first things first: There is a script that gives us needed info, as well as a Set-Cookie header. The cookie is: - X-V-RequestVerificationToken(hashtag){some id}={long id with a hashtag at the end}; expires={7 days}; path=/; HttpOnly And the script looks like this: ```js <script> var VParam = { appName: 'Dziennik VULCAN', moduleName: 'Panel ucznia', antiForgeryToken: '{antiForgeryToken}', appGuid: '{appGuid}', appUrl: 'https://dziennik-uczen.vulcan.net.pl', appCustomerDb: '{symbol}', apiUrl: 'https://dziennik-uczen.vulcan.net.pl/{symbol}/api', wiadomosciUrl: 'https://dziennik-wiadomosci.vulcan.net.pl/{symbol}/LoginEndpoint.aspx', wiadomosciNazwa: 'Wiadomości', logoutUrl: 'https://dziennik-uczen.vulcan.net.pl/{symbol}/LoginEndpoint.aspx?logout=true', user: { name: '{login}' }, serverDate: '{login date, but from the server}', appVersion: '{app version, 25.02....}', downloaderUrl: 'https://dds.vulcan.net.pl/Download/', // wtf is this downloaderBuffer: '500', downloaderInterval: '1', privacyPolicyUrl: 'https://www.vulcan.edu.pl/strona/polityka-prywatnosci-190', respectyPolicyUrl: 'https://dziennik-uczen.vulcan.net.pl/{symbol}/CookiePolicy/SzanujemyPrywatnosc', cookieSettingsUrl: 'https://dziennik-uczen.vulcan.net.pl/{symbol}/CookiePolicy/UstawieniaCookie', infoEnclosureUrl: 'https://www.vulcan.edu.pl/strona/klauzula-informacyjna-868', accessDeclarationUrl: 'https://www.vulcan.edu.pl/strona/deklaracja-dostepnosci-869', helpUrl: 'https://www.bazawiedzy.vulcan.edu.pl/bazawiedzy.php/wyszukaj/1/7/8/2/2/', manualUrl: 'https://aplikacje.vulcan.pl/uonetplus/DziennikUczen.pdf', lastUpdatesUrl: 'https://www.vulcan.edu.pl/strefa-klienta/aktualizacje/E-dziennik-UONET,7', footerLogo: '', isStudent: true, isParent: false, isDziennikVulcan: true }; </script> ``` Well, this is new... ### Doing requests Every API request requires two headers: - x-v-appguid: {appGuid in script} - x-v-requestverificationtoken: {token from script} Surprisingly enough, we can get the OLD info fields from another API request called "context", which gives us all the stuff, even the stupid "isArtystyczna13", etc... The request looks like this: GET - `https://dziennik-uczen.vulcan.net.pl/{symbol}/api/Context` (just our typical cookies from the website, no appguid or requestverificationtoken) - EfebSsoCookie - UonetPlus_SLACookie - UonetPlus_ASP.NET_SessionId - Long X-V-RequestVerificationToken from the Set-Cookie header in the panel website ```json { "uczniowie": [ { "idJednostkaSkladowa": {number, low}, "idDziennik": {diaryId}, "rodzajDziennika": 1, "dziennikDataOd": "2024-09-01T00:00:00+02:00", "dziennikDataDo": "2025-08-31T00:00:00+02:00", "isUczen": true, "isPrzedszkolak": false, "isWychowanek": false, "key": "{key in string, used in EVERY request as a query parameter}", "uczen": "{user's first name}", "oddzial": "{class, like 1-5}", "jednostka": "{school name}", "jednostkaGodzinaOd": null, "jednostkaGodzinaDo": null, "isDorosli": false, "isPolicealna": false, "is13": false, "isArtystyczna": false, "isArtystyczna13": false, "isSpecjalna": false, "pelnoletniUczen": false, "opiekunUcznia": false, "wymagaAutoryzacji": false, "posiadaPesel": true, "aktywny": true, "globalKeySkrzynka": "{message globalbox key}", "config": { "isOplaty": false, "isPlatnosci": false, "isZaplac": false, "isScalanieKont": false, "isJadlospis": false, "isOffice365": false, "isSynchronizacjaEsb": true, "isDydaktyka": true, "isNadzorPedagogiczny": true, "isZmianaZdjecia": true, "isZglaszanieNieobecnosci": true, "isLekcjeZrealizowane": true, "isLekcjeZaplanowane": true, "isPodreczniki": true, "oneDriveClientId": "{onedrive}", "projectClient": null, "payByNetUrlForPayment": "{paybynet stuff}" } } ] } ``` For things in the API that are fetched at the panel itself, it doesn't use the key, but in fetching grades, etc. it does. An example is "PlanZajecTablica": GET - `https://dziennik-uczen.vulcan.net.pl/{symbol}/api/PlanZajecTablica?key={key from context}` ```json [ { "data": "9999-99-99T00:00:00+02:00", "godzinaOd": "9999-99-99T00:00:00+02:00", "godzinaDo": "9999-99-99T00:01:00+02:00", "prowadzacy": "{teacher name}", "prowadzacyWspomagajacy1": null, "prowadzacyWspomagajacy2": null, "przedmiot": "{subject name}", "podzial": "{group in gr_(number) format}", "sala": "{room number}", "pseudonim": null, "zmiany": [], "zmianyUwagi": [], "adnotacja": 0, "dodatkowe": false, "zrealizowane": true, "idJednostkaSkladowa": {stuff from context} }, // ... ] ``` # I will continue research / dropping more info in the thread below - along with API requests that I have access to
val self-assigned this 2025-05-17 23:36:48 +02:00
Author
Owner

This is actually very similar to how the previous LoginType things (ADFS, etc.) do it, but with some extra requests needed to be done to get the same amount of information that is used in every day SDK usage.

As far as I know, this system does not support the HEBE api (no dostep mobilny) which means that this is the only way of integrating it with the SDK. :/

This is actually very similar to how the previous LoginType things (ADFS, etc.) do it, but with some extra requests needed to be done to get the same amount of information that is used in every day SDK usage. As far as I know, this system does not support the HEBE api (no dostep mobilny) which means that this is the only way of integrating it with the SDK. :/
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference: wulkanowy/sdk#4
No description provided.