Recording Some Daily Thoughts and Reflections
2023.09.03 The Magic of Running
Today is Sunday. I am usually a real homebody, so today, as usual, except for going out for meals, I spent most of the day lying in the apartment. However, this afternoon, I felt like I had been lying down for too long, so I got up and wrote a blog post. I spent a couple of hours organizing my thoughts and writing the introduction, and then I got ready to go downstairs for dinner. After dinner, I wanted to continue writing when I got back, but I don't know if it's because I had napped too long in the afternoon, I felt really uncomfortable all over - I had a headache and nausea. I checked my temperature and found that I didn't have a fever, but the symptoms persisted. It was very uncomfortable to lie down.
For the past two months, I have been going to the park to run every other day. Today was a running day, but I felt so unwell that I doubted if I could run at all. After lying down for a while, I felt that moving around might make me feel better. So, instead of my usual 9 o'clock, I went down at 8 o'clock. When I got to the park, I didn't start running right away. I took a leisurely walk around first, then started running. During the run, I made sure not to run too fast, tried to sweat more, and be more steady. After finishing the run, I walked for two laps. Although the symptoms had somewhat eased, they did not disappear completely. I didn't dare to go back and lie down again, so I sat on a park bench for a while. After about 20 minutes, miraculously, the symptoms disappeared. It seems that regular exercise is important after all - one's health comes first.
2023.08.27 The Character of a Product
I want to talk about something interesting here - the character of a product. Today, I started thinking about this after chatting with my classmate K about an update to the Shanco Mini Station, which reminded me of our team leader's blog: the official website of Effie, like a poem, is very simple overall, but with the embellishment of poetry, it gives the product character and depth. So, why does our product seem to lack a unique character? A colleague commented that perhaps it lacks the infusion of the founder's character.
Going back to the Shanco Mini Station, I basically developed it on my own, so I guess I am the founder. The Shanco Mini Station, as a campus mini-program I developed before, has about 55,000 users and a daily activity of around 12,000 users, with around 50,000 page views. The daily activity at the school is already quite high. Recently, I've been thinking about whether to continue updating its features, because if it needs filing now, the Shanco Mini Station Plus probably won't make it through, and I need to transfer the capabilities of the Plus to the Mini Station. K hopes to pass down the Mini Station so that others can continue to participate and make it even better. But then we discussed how previously well-performing campus mini-programs basically disappeared once the founders left. This reminded me that the home page of the Shanco Mini Station features a daily Chinese and English sentence, as well as an accompanying picture. The content I inject into the product as the founder, perhaps, is its character. Many classmates have told me that seeing a new sentence and picture every day is very comforting.
Looking at Effie, Effie is a rich-text note-taking application similar to Markdown, which is what I am currently working on. So, why doesn't the product I am making have its own character, or any really unique features? Perhaps it really is as my colleague said - lacking the infusion of the founder's character. During the implementation of features, there are too many suggestions and feedback that constrain us. In the review of requirements, there are also a multitude of opinions injected, so our product might be an application that meets the needs of the majority, but it may not become a wonderful product. Of course, this is not a mistake, but I feel it's a bit of a shame. Although this is just my job, I hope to create a product that dazzles users and makes them feel that there's something wonderful about it. I hope that, in the process of using our product, users can slowly discover its characteristics. Borrowing a line from Lei, perhaps it's not quite fitting here, but I always believe that wonderful things are about to happen.
Going back to the original question, since we can't inject our character into the product like a founder, I also hope to inject some of my own ideas into the product. Right now, I'm thinking of something: I hope that users can complete an entire document without needing to touch the mouse, using only the keyboard. This sounds easy to say, but it's actually very difficult to accomplish. How can we let users know all the shortcuts without using too many tooltips, strike a balance, and make users remember these shortcuts, or slowly discover and be amazed by them? This is no easy task.
2023.08.15 Returning Home Empty-handed
Recently, I took a total of 3.5 days of annual leave and used 2 days to return home to deal with a certain matter, which can be summarized as having to handle some file-related issues. I had been contemplating how to handle this matter since June, and it wasn't until August that the file finally arrived at the Rizhao Talent Market. I finally could start dealing with it. In reality, the file had actually reached the local human resources and social security bureau two weeks prior, but it had not been archived. I called to inquire about it and they prioritized the processing for me. The service at the bureau was impeccable. Regarding this file, I needed to go to the town to obtain a certificate, then go to the city to get another certificate, and then return to the town to submit the documents. The town said that the submission of the certificate must be done in person, so I had no other choice but to return home to handle this matter. Because it is quite far from my home, taking the high-speed train one way takes almost a whole day. Therefore, I could only return home on weekends and took 2 days of annual leave. I returned to Rizhao on Saturday and back to Hangzhou on Tuesday. Additionally, since these government offices do not work on weekends and have limited work hours during the week (6.5 hours a day), I had to visit three different places in town on Monday to get this matter sorted out. In order to expedite the resolution of these issues, I made numerous phone calls before returning home to find out about the situation, mainly to the town, where I made at least 20 calls. On Monday, I arrived half an hour before they started working and waited outside. After I approached them about handling the issue, they unexpectedly said they couldn't do it. Even though they had told me on the phone that they could handle it, and I had taken leave to come in person, they said it was not possible. I was truly at a loss. I had taken leave, purchased a ticket, and gone to the trouble of returning home, only to be told that they couldn't help me. I was really frustrated. If they couldn't handle it online, I would have understood, but to refuse in person? This was truly baffling. Consequently, with no other options left, I had to return to Hangzhou empty-handed.
This trip back home really had its share of bad luck. First, when I returned home, I wanted to clean the algae from the wall-mounted filter in my fish tank. Due to a moment of oversight, I started the machine without water, which damaged the centrifugal pump. So, I had to buy a new one. Then, the new one didn't have a good seal. I used a caulking gun to seal it, and only afterward did I realize that I should have used the right type of sealant to prevent it from becoming loose when in contact with water. The loose seal would result in a lack of waterproofing in the tank. However, I was short on time and could only address this issue during the National Day holiday. Then, in the evening, I found my bed's small fan to be too noisy. I took it apart and applied some lubricating oil, but it didn't work. I later found out that the noise was coming from the fan's joint with the fixed bar, which had become freely movable, causing the wire to tangle. So, I directly applied the right type of sealant to it, but then the fan couldn't rotate properly. It was only after looking through some information that I realized the joint needed to turn unidirectionally to allow the synchronous motor to rotate in one direction. However, it was already broken, so I had to buy a new one again. Lastly, apart from the unsuccessful handling of the file-related issue, I wasted the entire weekend and 2 days of annual leave, making an already tight supply of leave even more scarce. Lastly, I usually can't find a direct ticket to go back to Hangzhou, as I need to transfer. However, this time, the high-speed train from Rizhao to Yancheng was unexpectedly delayed, by a total of 16 minutes. It was truly unprecedented. I had often seen delays of 2 minutes, but it was the first time I encountered a direct high-speed train delay of 16 minutes. My main concern was that another delay might easily delay my second transit, but I still managed to arrive in Hangzhou successfully at the end.
This journey back home was full of various misfortunes, leaving me feeling quite EMO. I felt particularly helpless, especially regarding the handling of the file-related issue. I had made so many phone calls and inquired about all the processes, only for them to tell me they couldn't help when I visited in person. I even approached their leader to address the problem, but the response was still negative. I was really exasperated. I had planned for so long to deal with this matter, yet in the end, it was not resolved. I wasted a tremendous amount of energy and time, and yet it was unsuccessful. It was at this point that I felt exceptionally helpless, wishing someone could help me. Additionally, I hadn't figured out what to do next. At a time like this, I just felt so miserable. Leaving my hometown so far away to work in Hangzhou, I really wished to find a job location that was only a two-hour drive from home, making it more stable and convenient for handling personal matters. Of course, life is full of unexpected setbacks, and failure is a constant companion in life. Regret is the norm in youth, as expressed in the poem by Fang Yue from the Song Dynasty: "The unsatisfactory events are usually eight or nine; can it be any different when discussing with others?"
2023.07.15 Maintaining Positive Feedback
Recently, I've been pondering a question: How can I continuously improve myself at work and maintain a positive feedback loop? Although I've been working for almost a year and a half, most of the time has been remote. It's only recently that I joined the company in person, making me practically a newcomer to the workplace. As a newcomer, I might be full of enthusiasm and motivation for my current job, as many things are new to me. Slowly resolving issues is indeed a process of self-improvement and a normal feedback during work. Now, the question is, if I'm still doing the same job five years from now, then compared to a new graduate, my job could easily be handled by them. My five years of experience might help me resolve problems faster than a new employee, but it won't bring much personal improvement or reward for me. In this process, my own positive feedback will continuously decrease, which is not a good sign. It might lead to internal conflict and exhaustion.
Actually, I don't have a good answer to this question at the moment, and I might be thinking too far ahead. Personally, my main task for now is to build up momentum and broaden my horizons. Currently, I'm making a lot of progress in my work and receiving positive feedback. Looking ahead, as a newcomer, I see two main future development directions: if I pursue a managerial role, I hope to continually receive positive feedback on my management and business capabilities, which means continuously improving my problem-solving skills, not only technically but also in terms of business, communication, and vision. It's a comprehensive development direction. If I choose to pursue a technical path, then the focus of my problem-solving abilities and positive feedback would mainly center around my technical skills. For front-end development, there are two main directions: one is the broad front-end direction, including but not limited to Web, App, mini-programs, WebRTC, WebGL, and the other is the full-stack direction, leveraging Node.js to continuously take on backend work. Apart from working as a full-stack developer in the business domain, other directions such as Serverless might also be good development targets. Lastly, I have my own opinion - usually, the backend team understands the business the best, so the front-end team actually understands the product the best, and perhaps moving towards a product manager role could also be a good choice.
2023.06.30 Officially joined ByteDance
Yesterday, I officially joined ByteDance, and my seniority was reset to zero after joining officially, which feels a bit strange considering I interned for 1.19 years and now it's 0.0. So, my internship at ByteDance has come to an end, and now it's time to start working formally. At the moment, I haven't had any particularly unusual experiences with work because today is only my second day on the job. It feels like there's not much of a difference from the internship. Of course, this is largely because I interned for such a long period, and I'm already familiar with the work that has been assigned. Even though I'm a newcomer with a seniority of 0.0 years, I'm still responsible for the previous work, which seems a bit unfair. It's a bit challenging to not only start but also to shoulder so much responsibility.
One thing that has led me to some thoughts is the issue of renting a place. This time, I found a place to share with a friend in Hangzhou through a real estate agent, who has been in Hangzhou all along, so I'm like a newcomer with my bag in hand. Most of the rental-related work has been handled by my friend, including property viewings, and we used the services of a real estate agency called Beike this time. The agency fee is half of the monthly rent. Previously, whether in Beijing or Shenzhen, I was very resistant to using real estate agents, especially in Beijing and Shenzhen where they charged a full month's rent as the agency fee. However, after going through the rental process this time, I feel that using a real estate agent may not be unacceptable because it did save us a lot of trouble. I had always thought that the real estate agent was only responsible for facilitating the transaction and not for anything else after the deal was made, but this agent helped us with a lot of things after we signed the contract, such as dealing with the contract, setting up the internet, and handling some property-related issues. Furthermore, they even negotiated the rent down by 100 RMB and included property management. We rented a two-bedroom apartment for a total of 3700 RMB, and although there might be a possibility that the agent and the landlord collaborated to put on a show, it did result in a reduction of 100 RMB from the price listed on Beike. This experience made me realize that using a real estate agent may not be unacceptable, but of course, only if they provide more than just facilitating the transaction. A real estate agent is more like a service that we pay for, which can save us a lot of trouble. Additionally, the agents seemed quite familiar with the application process for housing subsidies, even more so than I, as an employee, was.
Speaking of Hangzhou, it's really hot here. At my hometown (Rizhao), the temperature is usually around 27 degrees Celsius, but in Hangzhou, it soared to 36 degrees. It's scorching, and if I hadn't been to Shenzhen before and anticipated that the southern region would be very hot, I would have found it quite unbearable as a newcomer. Apart from the relatively cheaper cost of living, due to paying three months' rent upfront and the agency fee, splitting the cost with my friend, each of us had to cough up 8325 RMB at once, which was quite a financial burden. This lower cost is relative; compared to Beijing, where this price might be 3/2 or even double, the cost of living here is still quite high, almost comparable to that in Beijing. So, apart from purchasing some basic living necessities, I prefer to buy online, as online shopping is generally cheaper regardless of the location, and being in the Yangtze River Delta region, the delivery is also very fast. Although I haven't experienced any major discomfort so far (apart from the heat), and I've been sleeping well, feeling rooted in a new place, I still want to go back home. It just feels better at home - it's so comfortable that I don't want to leave my hometown and run so far away.
2023.05.20 Completed graduation defense
Today, I finally completed my graduate thesis defense, drawing my graduate school life to a close. After this, perhaps I will talk about my graduate school life as I have about my four years of undergraduate study. Here, I actually want to talk about the issue of being a fresh graduate, as a recent event has made me reevaluate my future plans and my identity as a fresh graduate, but due to confidentiality, I cannot elaborate too much. This issue has made me reconsider my future plans and the identity of a fresh graduate.
To be honest, I never plan to settle down in a big city. Whether I will want to or not in the future, I don't know. At least for now, I'm thinking of working in a big city for a few years, and eventually returning to my hometown in a smaller city. In smaller cities, working in the public sector is more comfortable. This is a leveraged issue - the smaller the city, the more comfortable the public sector is. Once I'm back home, can I find a stable job there? This needs to be considered, and even in Shandong, it's very hard to find relevant internet-related work. The issue of the public sector leads directly to the issue of fresh graduate identity. It's well known that the identity of a fresh graduate is crucial, whether it's a civil service position, a state-owned enterprise, a bank, or anything else. Being a fresh graduate carries a lot of weight. In many places, only fresh graduates are wanted, and they have many advantages that non-fresh graduates do not have, such as Beijing and Shanghai hukou. Especially Beijing hukou, it can be said to be a once-in-a-lifetime opportunity.
So back to myself, actually now I'm considering whether it's too late to go back home and join the state-owned system after graduation. Most likely, I will be working at ByteDance in Hangzhou after my defense. After losing my status as a fresh graduate, will it really be feasible to return home in a few years? What can I do at home? There's no industry at home. I might only be able to work in a factory or drive for a ride-hailing service. Of course, I also have some skills, like making and selling pancakes or repairing computers, to start a small business. However, this may be something for ten years from now. No one knows what the future holds. Actually, it's mainly because I, a pessimist, have little confidence in the future and lack the courage to move forward. It's a mentality issue, but I still hope for a better future. In fact, life is tiring. Countless people are pushing you forward, and there is no way to stop and take a break. If I can give up some things and live a little happier, why not?
April 22, 2023 - Why Pursue a Master\'s Degree
This is actually related to my recent trip to Hangzhou, and it also happens to be my 1.01-year work anniversary. If I continue with internships, half of my time as a postgraduate student will be spent in these companies, mainly Tencent and ByteDance.
People often ask me why I want to pursue a master's degree at a non-elite university. Most people, including myself, think that it doesn't have much practical significance. If you've read my article "To My Four Years of University Life," you probably already have an idea. Let me briefly introduce my university experience. I studied industrial design in my freshman year, which was in the Art College. We didn't study subjects like advanced mathematics, and I faced enormous pressure when I transferred to the Computer Science department in the second semester of my freshman year. This pressure came from various aspects, including the need for catching up on courses, pressure from the college, and a series of problems caused by so-called "field trips" at that time. Not only did I have to catch up with fundamental topics in engineering, but I also had to attend an overloaded schedule—I had up to 25 classes a week, and I managed to squeeze in up to 32 classes at one point. Many of my classes overlapped, making it impossible to attend all of them. Despite the pressure, I spent my entire sophomore year catching up on courses. Even during summer and winter breaks, I stayed at the school to work on projects. I'm very grateful to Lu, who guided us through these projects and became my way into the world of computer science. I completed catching up on courses by the end of my sophomore year. In my junior year, I didn't have much free time and started doing projects during my holidays. I started taking on freelance projects, handling everything from project negotiation, project implementation, design, front-end and back-end development, servers, to payment collection. I achieved a lot on my own, including covering all my expenses starting from the junior year, and I never asked my family for money ever since. I even bought gifts for my parents during holidays.
Reflecting on all this, let's return to the question of why I chose to pursue a master's degree at a non-elite university. Looking at my university experience, I think I've been quite engaged. I've worked on projects in various technical areas, including front-end and back-end development and setting up servers. I've also negotiated projects. Although most of them were freelance projects, they still carry some weight for a university student. In the latter semester of my junior year, I had to start looking for internships. I was very enthusiastic and applied to Alibaba, but unfortunately, my application didn't make it past the resume screening. I began to reflect on where I might have gone wrong. There were many issues, and I won't go into detail here. I came to a conclusion by myself at that time—I'm not good at in-depth technical research. However, it was already late, and without an internship, it would be difficult to get into a so-called big company. So, I made a decision—to pursue a master's degree. It was like trading time for space, because I'm not a quick learner and I don't have exceptional innate ability. It takes me a long time to understand certain knowledge. Then I planned out my life as a postgraduate student. My ultimate goal for postgraduate study was still to join a big company, and my first-phase goal was to intern at Tencent during the first summer of my postgraduate study. I had researched thoroughly and knew that Tencent accepted first-year postgraduate students for internships without distinguishing between summer and regular internships. I achieved this goal. My second-phase goal was to find another internship at a big company in the spring of my second year, and I reached this goal as well. My third-phase goal was to join a big company during the autumn recruitment, and so far, it seems I have achieved this goal. These are my career-oriented stage goals, but there are two personal development goals that I failed to achieve—relearning Java from scratch and completely refactoring the ShanKe small station project. It's a pity that I couldn't achieve these two goals. Time was exceptionally tight. Half of my postgraduate study time was spent in external internships, and I also had to work on my thesis. Although I'm in a professional degree program, I still had research pressure. I did end up publishing an article in SCI, and now I just hope to graduate smoothly. I wrote a 97-page, 63,000-word thesis. It was quite a daunting task.
So, was the decision to pursue a master's degree right? I dare not say. On the one hand, the job market was still very good when I graduated from my undergraduate program. You can refer to my experiences in the internship spring recruitment of 2022 and the autumn recruitment of 2022. I won't say much more, but even if I end up at a big company, how long can I stay successful? On the other hand, I'm also inspired by Tianrui's work experience. Could I, like Tianrui, gradually gain experience without pursuing a master's degree and eventually make it to a big company and become competent on my own? It seems like everything comes down to choices. In reality, pursuing a master's degree is equivalent to spending three years trading work for a paper degree. How many postgraduate students have a clear plan for the future? Of course, everyone is unique, and the world is not simply black and white, right or wrong. Every decision made can only provide additional reference points at the crossroads of future choices. Once a decision is made, the key is to do well on the chosen path.
On a different note, some people say that I'm so dedicated. Is it possible that coding is just my hobby, and I'm simply working on things I enjoy? If I were asked to write a thesis, I certainly wouldn't have the same drive, because writing a thesis doesn't bring me joy. I'm only doing it because I have no choice. In fact, I'm being compelled to publish an article in SCI to meet my graduation requirements.
2023.04.19 Trip to Hangzhou
Going to Hangzhou was no easy feat. The high-speed train ride took a whopping 7 hours, and let me tell you, it was an uncomfortable journey. I set out on the 17th, waking up at 7 in the morning, then catching a hitchhike to Qingdao West Station. I finally arrived at Shanghai Hongqiao Station around 14:41, and let me tell you, it was crowded. After switching trains to Hangzhou, I arrived there around 15:44 and finally made it to the company. Goodness gracious, what a hassle. The return was no less torturous. I departed from Hangzhou East Station at 12:38, reached Shanghai Hongqiao at 13:46, then took a full-fare ticket at 14:11 until finally arriving at Qingdao Station at 19:39. I then took the subway to Jinggangshan Road, caught a taxi back to the school, and it was already 9 o'clock. Still had work to do, with two important tasks that needed to be completed, and I worked until past 11.
I took the opportunity to explore Hangzhou, and let me tell you, Hangzhou was scorching! While it was 15°C in Qingdao, it soared to 32°C in Hangzhou! What a leap! But Hangzhou's subway system far outstrips Qingdao's. Rumor has it that taking the subway in Hangzhou is faster than catching a taxi. I have to say, it was quite impressive. The Wuchang subway station is only a kilometer away from Bafangcheng, surrounded by an array of eateries. The food options were absurdly plentiful. In the evening, I went with Tianrui to see West Lake, although we couldn't go down to touch the fish. We simply strolled around the edge, which was a slight disappointment, ha-ha-ha. This trip to Hangzhou confirmed my decision to commit to working there. With every reward comes risk, and I've already weighed the rewards and risks. It's settled, and there's no need for further consideration.
As I'm actually working remotely from the school, I couldn't expense this trip. I did, however, manage to stay at the hotel where Tianrui was on a business trip, ha-ha-ha. That hotel was exorbitantly expensive, the priciest one I've ever stayed in. I'm very grateful to Tianrui for giving me the experience. I also realized that my tenure at this company is now 1.01 years, already exceeding the average. According to online chatter, one is considered officially employed after seven months, but once I'm officially on board, my tenure will reset to zero, and I'll become a newcomer again. A certain classmate once called me a seasoned pro, so I've actually changed my signature to "Newcomer slicer" now.
2023.04.07 ChatGLM Trial
I've been thinking these past few days about tweaking a model for myself, especially since during my postgraduate studies, 1/3 of my work involved virus handling, disassembly, and decompilation, while 2/3 focused on natural language processing, primarily with TensorFlow + BERT. These skills have now found their utility. I also feel confident in applying them to our products, so I applied for a V100 32G card and attempted a simple fine-tuning. As Chinese models are still relatively scarce, I opted to try ChatGLM's 6B model.
It took me two whole days of running and debugging, but deploying it turned out to be quite straightforward. However, I did encounter a hitch with Tsinghua source's outdated weights, which prevented a successful run. The HuggingFace model and preprocessing code proved to be crucial. During the fine-tuning process, I paired it with LoRA, and it consumed around 19G of VRAM. I faced another challenge due to faulty code, which took me a long time to debug. After struggling, I eventually switched to an open-source implementation the next day, and it was finally successful after two hours of debugging.
As the testing was rather simple, and the constructed prompt wasn't comprehensive, I encountered some issues with the output processing. This underscored the importance of the prompt. Ultimately, I managed to steer it away from identifying as ChatGLM and successfully taught it new information, such as the capabilities of our products and similar products within the company. On the whole, it seemed promising. Apart from the significance of the prompt, the parameters of LoRA are equally crucial and require continuous tweaking. After tinkering for some time, it felt like I was back in my postgraduate alchemical lab.
2023.04.02 Large-Scale Pre-trained Models
Yesterday, I originally intended to dabble with a large-scale pre-trained language model, but Ly insisted on dragging me out, introducing me as a third wheel to his date. I couldn't help but feel out of place, but in the end, I went along, hadn't seen him in years after all. We went climbing at Dazhu Mountain in the afternoon. My, that mountain was truly daunting. It exhausted me, but I managed to summit and the rhododendrons were quite a sight.
Upon return, as I was preparing to rest, a thought struck me. It seemed that I didn't need to rush into working with large-scale pre-trained models. It was actually due to seeing Lgy's post in the group about the LLama + LoRA model that I had considered trying it out. However, I mulled over a key point: this model had just been released, and LoRA significantly reduced the GPU requirements for training. This could drastically cut training costs and potentially allow for increased parameter sizes with the same compute power. Would this pave the way for a proliferation of various large-scale pre-trained models in the future, akin to mushrooms after the rain? This is certainly something to look forward to. Furthermore, Chinese semantics lack clarity, so large corpora are necessary for pre-training. I'm hopeful that stronger open-source models will emerge in the future. With foundational models established, we could then fine-tune using SFT (Supervised Fine-Tuning), then proceed to generation tasks, potentially achieving outstanding results in vertical domains.
2023.03.24 Novel Writing
So, here's the thing. These models are all about parameter quantity, measured in B, and it seems like throwing more parameters at the problem is the answer. But just having a hefty number of parameters isn't enough. One critical factor in training high-performance models is ensuring that the input data represents features rather than noise, which might require significant manual involvement and comes with its own costs. However, looking at it from a different angle, using established models for data distillation is a viable approach.
Of course, competing against giants like ChatGPT is incredibly challenging. For now, training large-scale language models is more suitable for specific vertical domains. However, the pace of development in this field is rapid. Take, for example, Facebook, now Meta, releasing the LLaMa model in late February, with versions including 65B, 33B, 13B, and 7B. Stanford, leveraging OpenAI's API, acquired tens of thousands of data points from ChatGPT and, similar to data/model distillation, fine-tuned the 7B LLaMa model with impressive results. tloen also reproduced Stanford's paper using LoRA, reducing training from a setup with 4x 80GB memory A100 to a single 4090 GPU, even with minimal resource usage during code execution for generation.
Future developments are unpredictable, but enabling adaptation to larger parameter scales with reduced computational power and emphasis on parallel processing could attract more participants to the field. Eventually, creating an open-source model rivaling ChatGPT is not out of the question. Additionally, it's noteworthy that the Transformer architecture was proposed by Google, and both Google and OpenAI took different paths. Google pursued the autoencoder language model, exemplified by BRET - Bidirectional Encoder Representations from Transformers, while OpenAI focused on the autoregressive language model, as seen in the GPT - Generative Pre-trained Transformer series. Currently, GPT appears to offer better functionality, but the future remains uncertain. Their varied training methodologies also lead to differences in downstream tasks: BERT leans towards discriminative tasks, featuring multiple encoder Transformers, excelling at tasks like text classification, named entity recognition, and sentiment analysis, while GPT leans towards generative tasks, employing multiple decoder Transformers, excelling at tasks like text generation, dialogue systems, and machine translation. However, things change, and the future may bring forth more creative models. At present, this field is a likely candidate for the next technological breakthrough.
Economic crises typically result from relative overproduction and insufficient consumption, and technological advancements often play a role in stimulating economic recovery to some extent. Naturally, the factors contributing to economic crises are diverse, and the conditions for resolution are not static. Let's hope that this economic crisis can be resolved expediently, taking advantage of the current technological advancements.
2023.03.19 Shanghai or Hangzhou for Work
Given that our operations span three cities, I had the opportunity to choose between Beijing, Shanghai, and Hangzhou when joining the company. Hangzhou wasn't initially in the mix, but as our business expanded, new colleagues joined in Hangzhou, and our major products and business operations also shifted there. So, Hangzhou became an option. Initially, I intended to go to Shanghai. Back when I was applying for internships, my plan was to intern in Shanghai, but due to various reasons like the pandemic, I ended up interning in Beijing. Now, I want to stick to my original plan and head to Shanghai, but recently I heard about talent subsidies in Hangzhou, which has made me a bit unsure. Originally, moving to Hangzhou might have only offered cheaper rent, but now, with the added incentives, it's becoming very appealing. Nevertheless, my agreement specifies Shanghai, and there are a few other considerations which I can't discuss here, making the decision between Shanghai and Hangzhou quite a dilemma.
2023.03.11 Computer Repair Skills
Today I saw a picture of Zhe ge repairing a computer posted on his Moments. Seeing this picture, Master Cai recalled his own skill in this trade after many years. It reminded me that I was a member of the technology department in my freshman year. I've repaired countless computers, from disassembling for cleaning and upgrading to installing new operating systems. It's a skill, just like learning to repair cell phones. Combining that with my ability to make pancakes, I could easily open a shop when I go back home. I could repair computers and sell food, very nice.
Previously, many classmates and friends had the idea of settling down in big cities like Beijing and Shanghai. I think that's basically an impossible thing, haha. The sky-high housing prices and income disparity make it unattainable for ordinary people. My idea is quite simple, work in the internet industry for two years, then go back home and start a pancake stand. But today, Master Cai remembered his skill and realized that after going home, he could not only make pancakes but also repair computers and other electronics, which is a nice way to make a living.
Last night I couldn't fall asleep. It wasn't because I was conflicted about where to go. After all, I won't graduate until the end of June, and I still have a relatively long time to make a decision. The issue I was pondering was my personal approach to analyzing any problem comprehensively. When facing a complex decision, I tend to list out all the pros and cons of various options and compare them one by one. I assess whether each item is the best choice or whether it aligns with my desires, and I also consider the weight of each advantage or disadvantage, although all of this is quite subjective. I simply hope to ultimately make a decision. However, can this comprehensive approach truly help me make the best choice? From past outcomes, it seems not. So last night, I kept contemplating whether this thorough consideration really matters, or if it even has practical significance. This reminded me of a saying: no matter which option you choose, there will be regrets. Therefore, I don't have an answer to this question. Perhaps in the future, I'll continue to make decisions using this approach, but I won't expect it to be the best choice. It will simply help me make a decision, avoiding so-called analysis paralysis. Perhaps, as the saying goes, no matter which option you choose, there will be regrets. So, the only thing to do now is to make a choice, whether it's the most correct is not important.
I also discussed this issue with many classmates, and I found it quite interesting. One classmate strongly recommended that I go to Shanghai, mainly because the subsidies in Hangzhou are limited, and Shanghai has many worth-experiencing places. But I think to myself, with that much money, I could buy so much instant noodles. I'm not very interested in going out to play, so the environment and scenery are not within my consideration. Another classmate strongly suggested that I go to Hangzhou, emphasizing the lower cost of living there and the possibility of settling down, deeming it unreasonable to go to Shanghai. Interestingly, the viewpoint regarding talent subsidies was consistent with the former classmate's. It's only a few bucks in talent subsidies! I thought, "Wow, these instant noodles are not cheap! But genuinely, the lower cost of living in Hangzhou is indeed a huge advantage, although the idea of settling down is just nonsense. After working in the internet industry for two years, I can go back home and start a pancake stand."
There was also a classmate who was very direct, saying, "Stop overthinking and just look at the money, that's it." This person's perspective is really good—clear goals and a practical person. The opinions of my classmates didn't help me make a decision, but they did give me some other ideas. It seems like everyone's understanding of choice is not consistent, let alone the aspects and considerations of choice, which is quite interesting.